...

The goal of this proposal is to prepare a set of configuration and practices to have reproducible/verifiable builds at packaging time, both by enhancing java natural build behaviour and by removing some variability introduced by some Maven plugins (core plugins at first, but also in the Maven eco-system)

Sources of unreproducible bits

• Timestamps:
  • Timestamp in pom.properties generated by maven-archiver (MSHARED-494)
  • Timestamp in plugin.xml and plugin-help.xml descriptors generated by maven-plugin-tools-generator (MPLUGIN-326)
  • Timestamp in MANIFEST.MF (Bnd-LastModified) generated by maven-bundle-plugin
  • Timestamps in ZIP/JAR files: file last modification time/date in central directory and file entry headers + possible optional fields "X5455_ExtendedTimestamp" (PLEXUS-ARCHIVER-48)
  • Timestamps in generated javadoc HTML files (can be disabled with javadoc options "notimestamp" and "bottom")
  • Timestamps in bytecode generated from Groovy code (added by GroovyClassLoader.addTimeStamp())
• Username:
  • Username in MANIFEST.MF (Built-By) generated by maven-archiver (MSHARED-661)
  • UID/GID in tar file entries
• Ordering:
  • Order of the file entries in a ZIP/JAR file (depends on file system order)
  • Order of goals in plugin.xml generated by maven-plugin-tools (MPLUGIN-261)
  • Order of the methods of the ObjectFactory.java file generated by JAXB/xjc (JAXB-598)

Version numbers of some of the tools used to build the artifact are also added to the artifact itself (e.g. "Built-Jdk" and "Created-By" MANIFEST entries added by maven-archiver to record the exact JDK and Maven versions used to build). This does not make the build unreproducible (because you just have to use the same tool versions to generate the same artifact) but it makes it more difficult to reproduce because of that. Line endings is also a problem, and even if we could force given line endings for build-generated text files (MANIFEST, pom.properties...), it would be hazardous to try to change the line endings of the resource files.

The underlying problem is that the pom file does not capture all the configuration of the build environment: it includes the plugins used during the build with their version number, but it does not include the version of Maven and the JDK, the Operating System and the architecture used to produce the artifact, etc...

What can we do? (non-exhaustive list):

• Drop the Maven/JDK version numbers in MANIFEST: most of the time you should get exactly the same result with two different versions of Maven and/or JDK (if you keep the same major version number). But you have to restrain to a given OS family because of line endings.
• Keep the version numbers in the MANIFEST: there are not easily accessible here. Moreover the semantic is poor (e.g. there is no JDK vendor so you can have different results if you use OpenJDK/Oracle JDK/Eclipse compiler, no reference of the Operating System used for line endings).
• Add a "reproducible build bill of materials" in an external file. This is the way Debian took to manage reproducible builds: they record the "build environment" in an external ".buildinfo" file that has all the information required to reproduce the build environment. If the frame of Maven we can think of several ways to achieve that (non-exhaustive list):
  • Create a secondary artifact (e.g. *-buildinfo.xml) with the required information
  • "Patch" the published pom file to add properties with the required information, something like:

<properties>
  <maven.reproducible.build.maven.version>3.5.0</maven.reproducible.build.maven.version>
  <maven.reproducible.build.jdk.version>8u123</maven.reproducible.build.jdk.version>
  <maven.reproducible.build.jdk.vendor>openjdk</maven.reproducible.build.jdk.vendor>
  <maven.reproducible.build.arch>amd64</maven.reproducible.build.arch>
  <maven.reproducible.build.os>linux</maven.reproducible.build.os>
</properties>

What are the issues to solve?