Toolchains

This is an important issue for NMaven. For clarity, I would like to explicitly list the requirements I need for C#. Some of these requirements are met by the proposal.

1. There are two types of requirements we need: a toolchain for the Java maven plugins and a toolchain for the target platform. For example, I may have a maven plugin requiring JDK 1.5 with a target C# language of Mono compiler under .NET framework version 2.0.

2. Profiles, such as J2ME (MIDP/CDC, etc) or .NET compact, need to have an associated bootclasspath configurable within the toolchain configurations.

3. Each module needs its own toolchain. Say module A is building under .NET 1.1 and module B is building under .NET 2.0. I need to support both under the same build. Partially addressed in proposal '3. Project'.

4. We do need the platform requirements specifiable (but not required) within the pom.xml file. For example, I would like the option to say that this module requires .NET 1.1 framework. You raise this issue in Q2.

5. There is a notion of compilers as well as executables (such as schema generators, wsdl generators, code analysis tools, etc). There needs to be a way to define toolchains for both of these. This is addressed in the goal statement of the proposal.

6. There needs to be a fairly robust way of defining default values for the toolchain configuration. For example, I have a state machine that fills in missing requirements based on operating system and what it finds installed on the platform. 

The biggest deficiency I see within the proposal is the limited notion of 'instance'. For example, say I have two instances of the JDK: jdk14 and jdk15. Now say I want to add support for jikes. I have doubled my instances to jdk14-jikes, jdk14-sun, jdk15-jikes, jdk15-sun. Add in support for 1.3 and 1.6 and it multiples further. The J2ME case with numerous foundations and profiles would be far worse. 

So the notion of 'instance' or exact toolchain matching breaks down in everything but the simplest case. As an artifact provider, I may want to specify jdk15 but let the plugin choose the compiler vendor for the build developer based on a set of matching rules. To give an example of the problem, say the artifact provider specifies jdk15-sun as the toolchain instance. Now say a build developer - who only has jikes installed on their system - types 'mvn compile', the build will fail when it should not.

One way of solving this, is to introduce the concept of toolchain requirements (nothing more than a list of name-value pairs, which in this case may just be Java.JDK=1.5). The toolchain plugin then needs to match these requirements based on values of the toolchain (or platform) capabilities, specified within the toolchains.xml file. As mentioned above, you could have some state-machine implementation fill in the missing requirements prior to passing off to the component handling the toolchain matching. If the system finds jikes, it will choose jikes as the compiler; otherwise it will choose Sun's compiler - or vice versa, depending on the matching rules.

Thanks for comments.

1. "maven plugin requiring JDK 1.5" is out of scope for toolchains proposal. Toolchains are only "hints" for maven plugins to on what tools to use for it's execution. It cannot/shall not influence the runtime environment of the plugin itself.

2. the configuration section in toolchains.xml shall allow that to happen. On the runtime side of things, I expect to have a basic Toolchain interface that will be extensible by multiple special-purpose subinterfaces. The J2METoolchain can have the bootclasspath then. The J2ME plugin will then use this "type" of toolchain.

3. shall be working fine, the maven-toolchain-plugin can be configured for each project separately. It even allows for defining different profiles configuration within the same project. Eg. once shall be able setup a profile to build/execute tests with different version of jdk/...

4. I'm generally leaning towards a toolchains-plugin rather than doing a special section in pom.xml. That plugin will prepare all the toolchain instances for use by other plugins down the stream. The other plugins (compiler/surefire) need to handle the case when the toolchain is not available (either fail or probably just use some default - the running JVM for example or what is defined it the plugin's own configuration.

6. as mention in 4. I believe the plugin is responsible for correct fallback behaviour. So the decision priorities look like this.
a. Check the plugin's own configuration (as defined in pom.xml). if present, use it.
b. check for a toolchain of the type I understand and use. if present, use it.
c. Some sort of fallback behaviour. for compiler/surefire/.., use the JVM version that is used by the maven build itself. In the case of your C# plugin, try to figure out the right platform based on OS install.
d. If c. doesn't work, fail.

The final comment is indeed the hardest nut. The solution you propose sounds reasonable. The require-provide relationship should be flexible. Then we can get rid of the instance identification as it was just the poor man's solution to this problem anyway. It could look like this:

<toolchains>
    <toolchain>
        <type>jdk</type>
        <provides>
            <jdkVersion>1.5</jdkVersion>
            <!-- anything else that the JDKToolchain understands.. -->

            <!-- to allow to virtually anything we can have this: -->
            <id>custom foo.com JDK with tweaked bootclasspath</id>
        </provides>
        <configuration>
            <directory>/home/mkleint/javatools/jdk1.5.0_07</directory>
        </configuration>
    </toolchain>
<toolchains>

and on the toolchain-plugin's side.

<plugin>
    <artifactId>maven-toolchain-plugin</artifactId>
    <configuration>
        <toolchains>
           <jdk> 
              <jdkVersion>1.5</jdkVersion>
           </jdk>
         <toolchains>
    </configuration>    
</plugin>

Any content under the <jdk> element in toolchain plugin is not mandatory, the plugin will search in the available toolchains for the first successful match. So empty <jdk> can result in any jdk match (1.6, or 1.4), the jdkVersion filled in means only 1.5 gets matched. Ideally we should have the same version resolution in the JDKToolchain (and possibly other toolchains as well) as the enforcer-plugin has now.

Don't forget the maven-antrun-plugin that should also utilize the toolchain e.g. calling rmic.

i'm not sure how the antrun plugin could ever utilize toolchains in a generic manner.

I'd like to see a global configuration section of the toolchains.xml file. I need to add a lot of tools, each of which has a different executable name but the same bin directory.

Shane Isbell: Isn't the findTool() method in the toolchain solving your problem? In the JDK toolchain, I can lookup "java", "javac", "javadoc" tools within one toolchain instance..

The issue that I am running into is that certain executables may be repeated between different toolchains, like the nunitConsole configuration below. If there were a global configuration, I would not need to repeat it.

<toolchains>
    <toolchain>
        <type>dotnet</type>
        <provides>
            <frameworkVersion>2.0</frameworkVersion>
            <vendor>MICROSOFT</vendor>
            <id>.NET Framework 2.0</id>
        </provides>
        <configuration>
            <csharpCompiler>C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\csc</csharpCompiler>
            <nunitConsole>C:\Program Files\NUnit 2.4.6\bin\nunit-console</nunitConsole>
        </configuration>
    </toolchain>
    <toolchain>
        <type>dotnet</type>
        <provides>
            <frameworkVersion>2.0</frameworkVersion>
            <vendor>NOVELL</vendor>
            <id>.NET Framework 2.0 - Novell</id>
        </provides>
        <configuration>
            <csharpCompiler>C:\Program Files\Mono-1.2.6\bin\gmcs</csharpCompiler>
            <nunitConsole>C:\Program Files\NUnit 2.4.6\bin\nunit-console</nunitConsole>
        </configuration>
    </toolchain>
</toolchains>

I second this require-provide approach. To give you a hard example of what would be nice to have, consider you want a java compiler that checks for 1.4 API compliance and does lint checks as supported since sun jdk 1.5. So I'd want maven to run this command:

 /usr/lib/jvm/sun-jdk-1.5/bin/javac -Xlint:all \
   -Xbootclasspath:/usr/lib/jvm/sun-jdk-1.4/jre/lib/rt.jar

Do you consider finding the appropriate paths to be in the scope of this proposal?
Would I request a (JavaJDK=1.5, Vendor=sun) compiler and the (JavaAPI=1.4) bootclasspath as two different tools, and combine them on the command line?

Or would I rather request a (JavaAPI=1.4, JavaJDK=1.5, Vendor=sun) toolchain which has the bootclasspath configured in toolchains.xml? Then I'd definitely want some kind of cross-refernecing within the toolchains.xml in order to reuse values specified only once.

Or would I specify my requirements more like (JavaAPI=1.5, Lint=all) and some pattern matching solution in the toolchain configuration could find a version that supports lint and pass the correct argument appropriately? This pattern matching engine might even be used to search for /usr/lib/jvm/sun-jdk-${JavaJDK} in order to automatically adapt as new versions get installed.

re: Martin von Gagern.

I'm not sure who is "I" and what is "maven" in the sentence "So I'd want maven to run this command".

The primary purpose of the toolchains proposal is to unify maven-plugins usage of resources like JDK used without the need to configure each plugin separately. The configuration is done in one place only "maven-toolchains-plugin", the other plugins reuse it's configuration.

So the actual code that needs to execute your command line example above is the compiler-plugin or some other custom plugin. That one shall support toolchains, to share the use of jdk picked with surefire-plugin and others. compiler-plugin currently supports your scenario only in if you manually configure the compiler args with "-Xlint:all -Xboothclasspath....". So that means you (as project owner) configure the path to jdk 1.4 rt.jar. I don't think toolchains can help there. Actually it's specifically implemented now that users (project owners) can override the toolchains configuration within the domain plugins. Eg. if you want all plugins to use 1.5, but want to compile with 1.4 for example, you configure the compiler-plugin as you do now ("executable" parameter with path to javac I think) and it doesn't use the toolchain.

if your example is deemed main stream and supported usecase by compiler-plugin devs, I imagine we could enhance the jdk toolchain to provide boothclasspath and the compiler-plugin could use it. on the user level it would boil down to additional, optional configuration parameter within the toolchain.xml file. for most people it will return the rt. jar of the actual JDK defined, but one could override that to return something else. if that gets implemented, I'm not convinced a support for "cross-referencing for value reuse" is necessary.

Aside from providing access to tools and versioning of them, some scheme for providing source and target version is needed.

Source compliance level should for example be shared between the compiler plugin and plugins generating source, for example the plugin: http://mojo.codehaus.org/was6-maven-plugin/ejbdeploy-mojo.html#jdkComplianceLevel needs to access the target source level. Today this will have to be pulled from the compiler plugin configuration "manually" - and not through some common shared configuration/API.

This has also been discussed on IRC: http://dev.rectang.com/logs/codehaus/%23maven/20080723.html&nbsp; (5:17 PM and onwards).