Discussion
In the mailing lists we've been discussing how to restructure the codebase to make releases easier to do, see isis-users thread and isis-dev thread.
Interim Conclusions (as of 7am, 3 Dec 12):
- yes, to idea of independent, more granular releases (this means that separate modules so don't share common parent, ie are separate artifacts)
- yes, flatten the modules at least as an interim step
- refactor the groupId/artifactId's to aid understandability:
- all artifactIds have an 'isis-' prefix so that, when assembled together in a single directory (eg WEB-INF/lib), they are easily identifiable as being Isis'.
- all artifactids also specify the component type, otherwise would be confusion between isis-sql.jar: is it an objectstore, or is it a security impl, or what?
- prefer that artifactIds read well, ie "isis-nosql-objectstore" rather than "isis-objectstore-nosql"
- consolidate artifacts where makes sense
- go with a small set of groupIds, broadly reflecting nature of component
- combine core and runtime artifacts
Still being debated:
- do directory names have to follow artifactIds? I'd rather they didn't, instead use component-impl format so lists better in OS shell (ls, dir)
- which modules to retire, if any?
- do we move the separate modules into their own git repos?
Refactoring: Resultant GroupIds
The table at the end of this page summarises proposed refactorings for all the current Isis artifacts. But to cut to the chase, we would end up with the following groupIds:
org.apache.isis.archetypes org.apache.isis.core org.apache.isis.objectstore org.apache.isis.viewer
We also have these additional groupIds, though current only used for artifacts that are not yet released:
org.apache.isis.security org.apache.isis.profilestore org.apache.isis.progmodel
Refactoring: Resultant Directory Strcuture
The table at the end of this page summarises proposed refactorings for all the current Isis artifacts. If we do the refactorings, we end up with the following directory structure.
Some notes:
- I've deliberately shown this alphabetically so we can easily what this means in practical terms.
- As the table indicates, I am now proposing that we move the unreleased artifacts into their own subdirectory. This will make them easier to exclude via a Maven profile.
- Note: a "/*" suffix indicates additional sub-directories.
- The number of artifacts in the core is significantly reduced. Mostly, it is just core and runtime. The others are either 'default' implementations of components that we bundle into core (bytecode, objectstore, profilestore, security), or are for unit testing, integration testing, or our embedded webserver (a dev env utility only)
archetypes/
dnd-xml
scimpi-nosql
wicket-restful-jdo
core/
deployment/
applib
bytecode-cglib
bytecode-javassist
core # most of core; basically the metamodel, facet factories, default progmodel, and a runtime API
runtime # the default runtime impl (IsisContext service locator, default impls of components, objectstore API)
development/
integtestsupport # for end-users when writing their apps, also used internally for tck end-to-end tests
tck/*
unittestsupport # core's testsupport module, mostly for our own unit testing purposes
webserver # not in runtime because is only a dev env utility
components/ # all components are released separately from core (have org.apache:apache as their parent)
objectstores/
inmemory
jdo/*
nosql/*
sql/*
xml/*
profilestores/
inmemory
sql
xml
progmodels
groovy
wrapper
security/
noop
file
ldap
sql
viewer/
bdd/*
dnd
junit
restfulobjects/*
scimpi/*
wicket/*
retired/
monitoring
viewer/
html
As noted above, all of the pom.xml's for each of the components are separately releaseable (have org.apache:apache as their parent).
We might decide at a later date to move each of these top-level directories into their own git repo. If we do that, will probably want to use main artifactId, eg: isis-dnd-viewer.git, isis-jdo-objectstore.git etc.
Refactoring: Proposed Module Moves/Renames/Consolidations etc.
The table below summarises proposed refactorings for all the current Isis artifacts.
Type column:
- C = core, parented by "org.apache.isis:isis" (as currently)
- L = rolled up into parent with other modules, see Notes column for further details
- A = alternate implementation, not parented by org.apache.isis:isis (instead would define its own parent), thus separately releasable. Might move to own git repo
- A = edition, ie an archetype, not parented by org.apache.isis:isis (instead would define its own parent), thus separately releasable. Might move to own git repo
- X = excluded, parented by "org.apache.isis:isis" but not released (excluded through use of Maven profile). Might one day be promoted to being an alternate implementation.
- R = retired
- D = deleted
IN THE TABLE BELOW, I HAVEN'T YET UPDATED THE PROPOSED LOCATION COLUMN; SEE INSTEAD THE DIRECTORY LIST ABOVE
current |
current |
proposed |
proposed location |
proposed |
proposed |
|
|---|---|---|---|---|---|---|
groupId |
artifactId |
action |
(relative to root dir) |
groupId |
artifactId |
Notes |
o.a.i |
isis |
C |
. |
o.a.i |
isis |
|
o.a.i |
applib |
C |
core/applib |
o.a.i |
isis-applib |
|
o.a.i |
isis.core |
C |
core/core |
o.a.i.core |
isis-core |
"Source from submodules rolls up to this parent (packaging=jar not packaging=pom)" |
o.a.i.core |
commons |
L |
— |
— |
— |
Rolled up into oai.core:isis-core |
o.a.i.core |
metamodel |
L |
— |
— |
— |
Rolled up into oai.core:isis-core |
o.a.i.core |
progmodel |
L |
— |
— |
— |
Rolled up into oai.core:isis-core |
o.a.i.core |
testsupport |
C |
core/unittestsupport |
o.a.i.core |
isis-unittestsupport |
"Not rolled up because intended to be referenced with scope=test; renamed to distinguish from integtestsupport" |
o.a.i.core |
webapp |
L |
— |
— |
— |
Rolled up into oai.core:isis-core |
o.a.i |
isis.runtimes |
D |
— |
— |
— |
"Delete module due to flattening " |
o.a.i.runtimes |
dflt |
C |
core/runtime |
o.a.i.core |
isis-runtime |
"Source from submodules rolls up to this parent (packaging=jar not packaging=pom)" |
o.a.i.runtimes.dflt |
runtime |
L |
— |
— |
— |
Rolled up into oai.core:isis-runtime |
o.a.i.runtimes.dflt |
testsupport |
C |
core/integtestsupport |
o.a.i.core |
isis-integtestsupport |
Utilities for end-users to write integration tests using Isis (cf Arquillian); also used by tck tests internally |
o.a.i.runtimes.dflt |
webapp |
L |
— |
— |
— |
Rolled up into oai.core:isis-runtime |
o.a.i.runtimes.dflt |
webserver |
C |
core/webserver |
o.a.i.core |
isis-webserver |
Utilities for end-users to bootstrap Isis as a webapp |
o.a.i.runtimes.dflt |
bytecode |
D |
— |
— |
— |
|
o.a.i.runtimes.dflt.bytecode |
dflt |
C |
core/bytecode-cglib |
— |
isis-cglib-bytecode |
"Suggest no longer positioned as 'default' since JDO for example does not require this module." |
o.a.i.runtimes.dflt.bytecode |
identity |
L |
— |
— |
— |
"Rolled up into oai.core:isis-runtime; is the new 'default'" |
o.a.i.runtimes.dflt.bytecode |
javassist |
C |
core/bytecode-javassist |
o.a.i.core |
isis-javassist-bytecode |
|
o.a.i.runtimes.dflt |
monitoring |
R |
retired/monitoring |
o.a.i.monitoring |
isis-monitoring |
Suggest we retire this module |
o.a.i.runtimes.dflt |
objectstores |
D |
— |
— |
— |
"Delete module due to flattening " |
o.a.i.runtimes.dflt.objectstores |
dflt |
C |
core/objectstore-inmemory |
o.a.i.core |
isis-inmemory-objectstore |
"Suggest rename to be more descriptive removing 'dflt'" |
o.a.i.runtimes.dflt.objectstores |
jdo |
A |
objectstore-jdo |
o.a.i.objectstore |
isis-jdo-objectstore |
|
o.a.i.runtimes.dflt.objectstores |
jdo-applib |
A |
objectstore-jdo/applib |
o.a.i.objectstore |
isis-jdo-objectstore-applib |
|
o.a.i.runtimes.dflt.objectstores |
jdo-datanucleus |
A |
objectstore-jdo/datanucleus |
o.a.i.objectstore |
isis-jdo-objectstore-datanucleus |
|
o.a.i.runtimes.dflt.objectstores |
jdo-metamodel |
A |
objectstore-jdo/metamodel |
o.a.i.objectstore |
isis-jdo-objectstore-metamodel |
|
o.a.i.runtimes.dflt.objectstores |
nosql |
A |
objectstore-nosql |
o.a.i.objectstore |
isis-nosql-objectstore |
|
o.a.i.runtimes.dflt.objectstores |
sql |
A |
objectstore-sql |
o.a.i.objectstore |
isis-sql-objectstore |
|
o.a.i.runtimes.dflt.objectstores |
sql-impl |
A |
objectstore-sql/impl |
o.a.i.objectstore |
isis-sql-objectstore-impl |
|
o.a.i.runtimes.dflt.objectstores |
sql-tests-common |
A |
objectstore-sql/tests-common |
o.a.i.objectstore |
isis-sql-objectstore-tests-common |
|
o.a.i.runtimes.dflt.objectstores |
sql-tests-served |
A |
objectstore-sql/tests-served |
o.a.i.objectstore |
isis-sql-objectstore-tests-served |
|
o.a.i.runtimes.dflt.objectstores |
xml |
A |
objectstore-xml |
o.a.i.objectstore |
isis-xml-objectstore |
|
o.a.i.runtimes.dflt |
profilestores |
— |
— |
— |
— |
"Delete module due to flattening " |
o.a.i.runtimes.dflt.profilestores |
dflt |
C |
core/profilestore-inmemory |
o.a.i.core |
isis-inmemory-profilestore |
"Suggest rename to be more descriptive removing 'dflt'" |
o.a.i.runtimes.dflt.profilestores |
sql |
X |
core/extras/profilestore-sql |
o.a.i.profilestore |
isis-sql-profilestore |
|
o.a.i.runtimes.dflt.profilestores |
xml |
X |
core/extras/profilestore-xml |
o.a.i.profilestore |
isis-xml-profilestore |
|
o.a.i |
progmodels |
D |
— |
— |
— |
"Delete module due to flattening " |
o.a.i.progmodels |
dflt |
L |
|
o.a.i.core |
isis-progmodel-dflt |
Suggest simply roll up into core |
o.a.i.progmodels |
groovy |
X |
core/unreleased/progmodel-groovy |
o.a.i.progmodel |
isis-progmodel-groovy |
|
o.a.i.progmodels |
groovy-applib |
X |
core/unreleased/progmodel-groovy/applib |
o.a.i.progmodel |
isis-progmodel-groovy-applib |
|
o.a.i.progmodels |
groovy-metamodel |
X |
core/unreleased/progmodel-groovy/metamodel |
o.a.i.progmodel |
isis-progmodel-groovy-metamodel |
|
o.a.i.progmodels |
wrapper |
X |
core/unreleased/progmodel-wrapper |
o.a.i.progmodel |
isis-progmodel-wrapper |
|
o.a.i.progmodels |
wrapper-applib |
X |
core/unreleased/progmodel-wrapper/applib |
o.a.i.progmodel |
isis-progmodel-wrapper-applib |
|
o.a.i.progmodels |
wrapper-metamodel |
X |
core/unreleased/progmodel-wrapper/metamodel |
o.a.i.progmodel |
isis-progmodel-wrapper-metamodel |
|
o.a.i |
isis.viewer |
D |
— |
— |
— |
"Delete module due to flattening " |
o.a.i.viewer |
bdd |
X |
core/unreleased/viewer-bdd |
o.a.i.viewer |
isis-viewer-bdd |
|
o.a.i.viewer |
bdd-common |
X |
core/unreleased/viewer-bdd/common |
o.a.i.viewer |
isis-bdd-viewer-bdd-common |
|
o.a.i.viewer |
bdd-concordion |
X |
core/unreleased/viewer-bdd/concordion |
o.a.i.viewer |
isis-bdd-viewer-concordion |
|
o.a.i.viewer |
bdd-concordion-tck |
X |
core/unreleased/viewer-bdd/concordion-tck |
o.a.i.viewer |
isis-bdd-viewer-concordion-tck |
|
o.a.i.viewer |
dnd |
A |
viewer-dnd |
o.a.i.viewer |
isis-dnd-viewer |
|
o.a.i.viewer |
html |
D |
retired/viewer-html |
— |
— |
Suggest we retire this module |
o.a.i.viewer |
junit |
X |
core/unreleased/viewer-junit |
o.a.i.viewer |
isis-junit-viewer |
|
o.a.i.viewer |
junit-tck |
X |
core/unreleased/viewer-junit-tck |
o.a.i.viewer |
isis-junit-viewer-tck |
|
o.a.i.viewer |
restfulobjects |
A |
viewer-restfulobjects |
o.a.i.viewer |
isis-restfulobjects-viewer |
|
o.a.i.viewer |
restfulobjects-applib |
A |
viewer-restfulobjects/applib |
o.a.i.viewer |
isis-restfulobjects-viewer-applib |
|
o.a.i.viewer |
restfulobjects-viewer |
A |
viewer-restfulobjects/viewer |
o.a.i.viewer |
isis-restfulobjects-viewer-viewer |
|
o.a.i.viewer |
restfulobjects-tck |
A |
viewer-restfulobjects/tck |
o.a.i.viewer |
isis-restfulobjects-viewer-tck |
|
o.a.i.viewer |
scimpi |
A |
viewer-scimpi |
o.a.i.viewer |
isis-scimpi-viewer |
|
o.a.i.viewer |
scimpi-dispatcher |
A |
viewer-scimpi/dispatcher |
o.a.i.viewer |
isis-scimpi-viewer-dispatcher |
|
o.a.i.viewer |
scimpi-servlet |
A |
viewer-scimpi/servlet |
o.a.i.viewer |
isis-scimpi-viewer-servlet |
|
o.a.i.viewer |
scimpi-tck |
A |
viewer-scimpi/tck |
o.a.i.viewer |
isis-scimpi-viewer-tck |
|
o.a.i.viewer |
wicket |
A |
viewer-wicket |
o.a.i.viewer |
isis-wicket-viewer |
|
o.a.i.viewer |
wicket-model |
A |
viewer-wicket/model |
o.a.i.viewer |
isis-wicket-viewer-model |
|
o.a.i.viewer |
wicket-ui |
A |
viewer-wicket/ui |
o.a.i.viewer |
isis-wicket-viewer-ui |
|
o.a.i.viewer |
wicket-viewer |
A |
viewer-wicket/viewer |
o.a.i.viewer |
isis-wicket-viewer-viewer |
|
o.a.i.viewer |
wicket-tck |
A |
viewer-wicket/tck |
o.a.i.viewer |
isis-wicket-viewer-tck |
|
o.a.i |
security |
— |
— |
— |
— |
"Delete module due to flattening " |
o.a.i.security |
isis.security.dflt |
C |
core/security-noop |
o.a.i.core |
isis-noop-security |
Renamed for understandability |
o.a.i.security |
file |
C |
core/security-file |
o.a.i.core |
isis-file-security |
|
o.a.i.security |
ldap |
X |
core/unreleased/security-ldap |
o.a.i.security |
isis-ldap-security |
|
o.a.i.security |
sql |
X |
core/unreleased/security-sql |
o.a.i.security |
isis-sql-security |
|
o.a.i |
isis.tck |
C |
core/tck |
o.a.i.core |
isis-tck |
|
o.a.i |
isis.tck-dom |
C |
core/tck-dom |
o.a.i.core |
isis-tck-dom |
|
o.a.i |
isis.tck-fixture |
C |
core/tck-fixture |
o.a.i.core |
isis-tck-fixture |
|
o.a.i.skins |
classic-skins |
C |
core/site-skin |
o.a.i.core |
isis-site-skin |
Renamed for understandability |
o.a.i |
quickstart-archetype |
T |
archetypes/wicket-restful-jdo |
o.a.i.archetypes |
isis-archetype-wicket-restful-jdo |
each archetype represents an 'edition' of Isis; separately releasable |
o.a.i |
quickstart-archetype |
T |
archetypes/scimpi-nosql |
o.a.i.archetypes |
isis-archetype-scimpi-nosql |
|
o.a.i |
quickstart-archetype |
T |
archetypes/dnd-xml |
o.a.i.archetypes |
isis-archetype-dnd-xml |
|
Original text that initiated the discussion
At the moment it's a daunting task to fully verify all the components of the framework are working correctly prior to pushing out a release. I suppose that wouldn't be so much of a concern if we had better automated tests, but, hey, that's how things are.
Moreover, frankly, some of the modules are best considered spikes and probably don't constitute something that we would consider production-ready. Or, at least, they haven't been used in a real-world context, so it's not possible to say whether they are really fit-for-purpose. I include here quite a lot of the stuff that I have worked on over the last few years, eg the wrapper-progmodel, the groovy progmodel, probably also the JUnit viewer and BDD viewer. Now that we are a top-level project, I'd like it that the code we put out is thought of as production ready.
In order to make the framework easier to release, I propose that we use Maven profiles to be able to release subsets of modules, that a given contributor can stand behind and say: "yes, those modules are working and are fit for general consumption". Each release would consist of the certain core modules, along with selected additional viewers and object stores.
For example, we might have:
- v0.3.1 core + objectstore-jdo + viewer-wicket + viewer-restfulobjects // a "wicket/jdo release" - eg tested and released by Dan
- v0.4.0 core + objectstore-nosql + viewer-scimpi // a "scimpi/nosql release" - eg tested and released by Rob
- v0.5.0 core + objectstore-dflt + objectstore-xml + viewer-dnd // a "dev release" - eg tested and released by Rob
- v0.6.0 core + objectstore-jdo + viewer-wicket + viewer-restfulobjects // another "wicket/jdo release"
- v0.7.0 core + objectstore-sql + viewer-html // a "html/sql release" - eg tested and released by Kevin
- v0.8.0 core + objectstore-jdo + viewer-wicket + viewer-restfulobjects // another "wicket/jdo release"
Whenever a release goes out, we would update the site to indicate the most recent version of the components.
Now, when I say "release", what I actually mean here is the deployment of binary artifacts up to the Maven central repo. This, after all, is what most users/would-be users in the community would use and consider a release. However, Apache's own formal definition of "release" is actually the source code release ... this is what the VOTE mechanism is for. I don't see this changing... the vote basically says that the software complies with all the legal license stuff etc. The whole codebase is tagged with that release number, and the generated src.zip is a zip of this release.
One benefit of this is that it allows the deployment of other modules to be performed "after the fact". For example, suppose that I (Dan) puts out the v0.3.1 release as above, and then Rob later on tests the scimpi viewer in that tag and finds it works well. He can (I think) push the release of the scimpi viewer up to Maven central repo.
As I see things there are several benefits to this scheme:
a) (as already noted) the user community will only see binary releases that are known to be of production ready. This should mitigate the "is it safe to use this component" worry
b) (as hinted at) it should be possible for individual contributors to put out releases of their modules more rapidly
c) we get visibility of which modules aren't being released; for example, we might say that if a module hasn't been released by anyone for 18 months, say, then it's probably time to remove it from the codebase.