We should experimentally support building multiple projects in parallel support in Maven 3.0.
Superceded by Parallel builds in Maven 3
Outstanding Issues
In my earlier revision 833566, I attempted to fix MNG-3004 Allow build lifecycle to execute projects in parallel
The general consensus around this bug (which has 25 votes) is that multithreading support can't work correctly right now because of MNG-2802
Concurrent-safe access to local Maven repository
Proposal: Fix MNG-3004 Without MNG-2802
Several people have remarked in comments to MNG-3004 that it's appropriate to try to fix MNG-3004 (parallel projects) without fixing MNG-2802 (thread-safe local repo). In doing so, we'll allow users to optionally enable building multiple projects simultaneously. This is worth doing, because it can be tested in the wild, and because for some users, it will be immediately useful (e.g. especially if you use --offline mode).
I agree with these comments, and attempted to implement them in revision the MNG-3004 branch: http://svn.apache.org/repos/asf/maven/maven-3/branches/MNG-3004/
All integration tests now pass on my machine with the MNG-3004 branch, so I'd like to land it back in trunk again and cut a quick alpha with this additional feature.
Using the MNG-3004 branch, you can now do this:
mvn install -Dmaven.threads.experimental=4
It works on my machine. If it works for you great; if it doesn't, we can figure out what's wrong together.
A Note About Defaults
Currently the code defaults to maven.threads.experimental=0, in which case everything is done on the main thread. Once this code gets some testing, we should at least consider making "1" the default. This fires up a single executor thread who does all work; the main thread joins to wait for it to finish.
I think leaving the value at 1 is a good compromise between avoiding complex threading issues and exercising tricky code.
What's in the Future?
I think that this feature would benefit from more tooling to make parallel projects understandable. The logger should be enhanced to identify which projects are logging which messages, and the final output of Maven should report more clearly which projects had warning/error messages, and how many such messages were reported.
We should clearly fix MNG-2802 at some point. I've heard that the new jetty wagon should be thread-safe, so maybe the solution is just to fix the bugs there and get it rolled in.
Another possibility (which is also good in its own right) is implementing a settings.xml option allowing users to configure their local repository layout. This could allow users to choose an alternate implementation that is thread-safe.
Votes
+1:
+0:
-1: