Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Added some details of smart reactor

...

  • A proper place at the beginning and end of the build to execute. The AbstractMavenLifecycleParticipant is working now to do some of that work, but with aggressive concurrency many of the techniques currently used (like the deploy-at-end technique in the deploy plugin) are not thread safe and assume a mode of serial execution. Robert Scholte: We should think of a smarter lifecycle-mechanism.  The install and deploy phases are kind of finalize phases, which are only useful when all projects were packaged successfully.
  • Execution environments: being to specify versions of the JVM in a similar way as Eclipse does, for spawned processes being able to pass in all Maven parameters and settings consistent (MNG-5199). This probably requires coming up with the one way to fork/invoke projects and finally get rid of the 5 ways we currently do it. Here I'm not sure what happens to the toolchains as they are a mechanism which is primarily used for finding a JDK other than the one Maven is started with.
  • Overhauling profiles
  • Smarter reactor. If you have a reactor with four projects: common-parent, jar-module, war-module and acceptance-tests-module. There is a dependency chain within the reactor, e.g. acceptance-tests-module:pom depends on war-module:war depends on jar-module:jar. If the user invokes mvn test then Maven 2/3 will do the wrong thing, either blowing up because there is no war-module:war dependency available, or using the war-module:war that was installed 5 weeks ago when somebody last ran mvn install. Neither of these is the correct thing to do. The correct thing to do is something along the lines of the following. 
    • The build plan starts with all modules trying to attain the test phase. Then we start inspecting dependencies required by each module. 
      • jar-module has no dependencies, so it stays as is. 
      • war-module has a plugin bound to the compile phase that needs either jar-module:jar or jar-module/target/classes, and as the latter is produced by jar-module in the process-classes phase we add a constraint that the jar-module process-classes phase must happen before the war-module compile phase.
      • acceptance-tests-module has a plugin bound to the generate-test-resources phase that needs war-module:war. We add a constraint that the war-module package phase must happen before acceptance-tests-module generate-test-resources phase and consequently the war-module is now escalated to build as far as the package phase.
      • war-module has a plugin bound to the package phase that needs jar-module:jar. We add a constraint that the jar-module package phase must happen before the war-module package phase and consequently the jar-module is now escalated to build as far as the package phase.
    • The net result:
      • jar-module process-classes happens before war-module compile
      • jar-module package happens before war-module package
      • war-module package happens before acceptance-tests-module generate-test-resources

    • A single threaded reactor will run jar-module up to package, war-module up to package and finally acceptance-tests-module up to test
    • A parallel reactor with three threads will be able to put locks in places such that the reactor behaves correctly
    • This would be quite a big change, but one that makes Maven a lot more useful.
      • Need a way to flag a packaging type as being produced by a mojo execution
      • Need a way to flag where a mojo can take dependency substitutions... this is to let the mvn compile use case work without running the tests of upstream modules within the reactor. Another way to achieve that is to reorder the lifecycle and move the test phases after package... of course if we start assuming fork-join lifecycles we could let the package phase run in parallel with the test phases which would let us split things up... another option would be to let phases be marked with constraints... so we can define a natural order to the lifecycle (i.e. the order the phases are listed in) and add phases that must happen before to the phase definition.
<phases>
<phase>validate</phase>
<phase after="validate">initialize</phase>
<phase after="initialize">generate-sources</phase>
<phase after="generate-sources">process-sources</phase>
<phase after="initialize">generate-resources</phase>
<phase after="generate-resources">process-resources</phase>
<phase after="process-sources">compile</phase>
<phase after="compile">process-classes</phase>
<phase after="initialize">generate-test-sources</phase>
<phase after="generate-test-sources">process-test-sources</phase>
<phase after="initialize">generate-test-resources</phase>
<phase after="generate-test-resources">process-test-resources</phase>
<phase after="compile, process-test-sources">test-compile</phase>
<phase after="test-compile">process-test-classes</phase>
<phase after="process-test-classes">test</phase>
<phase after="process-resources, compile">prepare-package</phase>
<phase after="prepare-package">package</phase>
<phase after="package, test-compile" finally="post-integration-test">pre-integration-test</phase>
<phase after="pre-integration-test">integration-test</phase>
<phase after="integration-test">post-integration-test</phase>
<phase after="post-integration-test">verify</phase>
<phase after="verify, package">install</phase>
<phase after="install">deploy</phase>
</phases>

The sequence of phases in the lifecycle would define the natural order which is what applies when the phase is specified on the command line. The after constraints would only apply when escalating a module, so when we escalate the war-module to package, we see that it must happen after prepare-package, which is after compile and process-resources. Now that would not have any net effect when running mvn test on the example reactor, but it would have an effect when running mvn test-compile as it would mean that any unit tests in jar-module and war-module would not be executed.

This kind of extra lifecycle constraints would also help us parallelise the build even more.

      • Finally, if we can find a way of flagging mojos as producing artifacts with particular packagings, Maven could also escalate across different lifecycles. Say for a contrived example we have the site lifecycle producing the javadoc:jar, if we have the dependency plugin try to unpack the javadoc:jar into the war file during prepare-package then when invoking package we could insert a happens before constraint such that the pre-site phase gets pulled in.
      • Also, might be helpful to have pre-site require validate as the first phase and perhaps site requires process-sources and process-test-sources which would remove a lot of the forked lifecycle issues with e.g. javadocs when running mvn clean deploy site-deploy. You would still need a forked lifecycle type of thing for code coverage, but most of the other use cases would then be redundant.

Cleanup

  • All the listener interfaces are pretty convoluted and incomplete, we should probably review again. EventSpies cannot be hooked into the build as normal extensions so they are really kind of useless (hence the recent addition of AbstractMavenLifecycleParticipant#afterSessionEnd)
  • ReactorManager: there are many inconsistencies where test jars are not resolved in the reactor, and the special magic that happens in the compiler plugin currently where it sets the artifacts file to the compile classes directory. This all needs to be encapsulated in a new workspace reader that behaves consistently and doesn't require special behavior in certain plugins. If you wrote a new compiler plugin for example and didn't set the file everything would break which is just poor encapsulation.

...