"Make" like build behavior mode

Maven currently has a top down build approach where you start at the top of a reactor and build all children. One of the most common requests I get from my developers is an easy way to build certain artifacts from the bottom up. Often times a build, especially large ones, will contain many modules needed for a "full" build, but are actually made up of pockets of interdependencies. It's not always possible to logically group these all together with a common parent to enable easily building a subtree.

For example, you may have a project comprised of services, ui's and packages: 

+---packages
|   +---a-package
|   +---b-package
|   \---c-package
+---services
|   +---a-service
|   +---b-service
|   \---c-service
\---ui
    +---a-ui
    +---b-ui
    \---c-ui

The packages inherit from the package parent, etc. Assume that "A-package" depends on "a-service" "b-service" and "a-ui"

In Maven, there is currently no easy way to make a change to "a-service" and build it and the package at once. This can be controlled to some extent with modules in profiles, but this quickly becomes unwieldy and doesn't support the granularity needed.

 Out of Scope

It is out of scope for this proposal to determine if a project actually needs to be rebuilt based on the contents. (ie checking if anything has actually changed) This is simply intended to be an extension to the reactor behavior in choosing which projects should be included in the reactor. 

Proposed Solution 1 (NOT IMPLEMENTED) 

The ideal use case for this issue is:

1. Developer makes change to code in "a-service"

2. Developer goes to "a-package" and executes "mvn -A install"  (-A for all)

3. Maven walks up the parent tree to see how far up the source goes. Then any dependencies in the graph that are found in the contained source on disk are ordered and built. Everything else is ignored in the build.

Alternate Use Case 2:

2. Instead of going to  "a-package" and executing mvn, the developer goes to the top level parent and executes "mvn -Apackages/a-package" (in this case defining the module that should be considered for dependency graphing)

3. Maven builds the graph and builds what is needed.

This use case isn't ideal but is probably easier to implement since the top level parent doesn't need to be located and everything to be built is included in the subtree. 

Porting maven-reactor-plugin

The maven-reactor-plugin can be used today to build stuff. We'd like to port at least reactor:resume, reactor:make, reactor:makeDependents into core. I'm thinking it would go something like this:

New options:

  • -rf --resume-from: Resume reactor from specified project
  • -pl --project-list: Build the specified reactor projects instead of all projects
  • -am --also-make: If a project list is specified, also make projects that the list depends on
  • -amd --also-make-dependents: If a project list is specified, also make projects that depend on projects on the list

Examples:

  • resume: mvn -rf packages/b-package
  • make: mvn -am -pl ui/a-ui,ui/b-ui
  • make dependents: mvn -amd -pl packages/a-package,packages/b-package
  • make both: mvn -am -amd -pl packages/b-package

At least at first, core would not include the new reactor:makeScmChanges because that would require maven-core to depend on maven-scm.

  • No labels

6 Comments

  1. John Dennis Casey

    This sounds like a pretty nice feature to have. It's sort of a more focused alternative to the standard reactor. Would you allow multiple "hint" projects, and allow collection/building of only the projects related to them?

    Also, while I do like this idea, long-term I'd much rather see a generalization of these two types of reactor into a pluggable reactor component. This component would be responsible for tracking down all the projects relevant for the current build (given some hints like the a-package one above), and then returning the set of projects (or even, pom files) to Maven for building.

    This would allow us to implement true build-on-demand functionality, where the entire dependency tree is resolved from pom in the repository, then pulled out of SCM to some workspace location...and built all as a single reactor. This is a use case that's come up a few times, especially in CI discussions, and if we're going to get into the business of alternate project collections, we should consider opening the mechanism up for future development.

    However, all that grandiose detail can wait for 2.2. (wink)

  2. Unknown User (gagern)

    Nice! I think I prefer the second use case, as this gives you more control over the scope of the rebuild, as you could call this somewhere on an intermediate level of a multi-level hierarchy, in order to only consider dependencies completely within that part of the tree, and you could also specify multiple leaves as modified, and have the modules depending on several of them built only once.

  3. Daniel Fabulich

    I think we have to go with the second "alternative" use case; in fact, I think implementing the first use case may be impossible.

    Remember, there are three notions of parent to keep distinct here:
    1) The parent directory
    2) The aggregator POM (which includes <modules>)
    3) The parent POM (declared in a <parent> element in the children)

    While the parent POM (2) is usually an aggregator POM (3) and usually lives in the parent directory (1), these three notions of "parent" don't have to be the same at all. The parent POM may be in any directory, even on another machine (and pulled down from a remote repository). Similarly, the aggregator POM need not be a parent POM and need not be in a parent directory: it can be in a sibling directory. In that case, the module list may say something like: <module>../foo</module><module>../../bar</module>

    The aggregator POM can be literally anywhere on the machine; in the worst case we'd be forced to scan the entire filesystem for pom.xml files, which is unacceptable.

    Note that it is possible for child projects to give Maven a hint as to the location of their parent POM, but it's not currently possible to tell Maven where one's aggregator POM might be. This is made all the worse because you can in fact have multiple aggregator POMs, some of which only aggregate a given subproject under certain conditionally activated profiles.

    That's not bad, though, because even if the primary use case were possible, as Martin points out it wouldn't be as nice as the alternative use case.

  4. Daniel Fabulich

    I made a slight modification to Use Case 2, which used to just say "-Aa-package". I think we shouldn't support that: we should require the user to specify "-Apackages/a-package".

    Specifically, there may be two artifacts in the reactor with the same artifactId but different groupIds, e.g. foo/war and bar/war. If we allow the user to just say "war" then we need a mechanism for handling ambiguous declarations.

    If we force the user to specify a relative path and give a clear error message ("No such directory: /home/person/project/war") then the user can figure out what needs to happen.

  5. Jason van Zyl

    Do you have a mode where a user may be in a leaf node, and without specifying any projects Maven just builds any of the projects required? Basically you could walk up the tree until you have no parent on disk and consider that your reactor. So that I could step into a child project and just type "mvn <option> install" and the right thing would happen: walk up a directory and build the right projects depending on the option. By default can we not have to specify any projects?

    I also think "mvn -rf" is probably not the best choice of short cut given the "rm -rf" means bad things ensue.

  6. Daniel Fabulich

    No, currently there's no mode that allows you to sit in a leaf and make projects you depend on.

    Duh, you're right, -rf is probably a dangerous choice. Other suggestions? -R and -r are both taken, so I think it has to be two letters. -rp for resume project?