Release Guide

The process for releasing Apache Crunch works as follows:

1. Nominate a Release Manager
2. Prepare for Release
3. Create a Release Candidate
4. Run the Vote
5. Publish the Release
6. Announce the Release
7. Clean up

Nominate a Release Manager

The committers agree on a release manager who is responsible for executing the process described in this document. The release manager ...

• must be a committer of the project
• needs a PGP key, preferrably linked to the Apache web of trust (see instructions for creating one)
• needs a settings.xml ready to deploy to Apache's Nexus instance (see instructions)

Prepare for Release

• Decide what goes in the release and set the resolved version field of all affected issues in JIRA accordingly
• Create a release branch (via maven-release-plugin) as soon as all issues for the version in question have been marked as fixed
• Check licenses: Watch out for new dependencies; don't forget to check LICENSE/NOTICE files in our JARs
• For all files added between releases, make sure they have correct license headers
• Prepare release notes based on JIRA (Note: we don't do that yet)

Create the Release Candidate

Currently, Crunch publishes two convenience binaries: one against the hadoop1 APIs, which will be released as Crunch version X.Y.Z, and a second against the Hadoop 2 APIs that is given the version X.Y.Z-hadoop2. There is still only a single source release for each version. The ordering of these steps is important for creating a valid release candidate.

First, clone a fresh repository (just in case) and create the release branch if there isn't one already:

  mvn release:branch -DbranchName=apache-crunch-X.Y

Note that the version number has two digits, not three! When prompted for the working copy version, increment Y by 1 and set Z to 0. Check the change to your working copy and push it to master:

  git push origin
  git checkout apache-crunch-X.Y

Make sure everything is in order, make any required last minute changes, then do a dry run of the release tagging process for Hadoop2:

  mvn release:prepare -DdryRun=true -Dcrunch.platform=2

You'll be asked for the release version, the tag name, and the next development version. The release version should be X.Y.Z-hadoop2, the tag name and the next development version can be anything because we're going to repeat this process in a few minutes for hadoop1. Once you verify that everything works properly, prepare the release for hadoop2 for real:

  mvn release:clean release:prepare -Dcrunch.platform=2

Examine the release branch and the tag to be sure everything worked (make a diff), then perform the release (which needs the property files created during release:prepare):

  mvn release:perform -DlocalCheckout=true -Dcrunch.platform=2

This deploys the hadoop2 Maven artifacts to a staging repository in Nexus. It will also create source and binary release artifacts in crunch-dist/target (ignore the stuff in target/, it's from the ASF POM and we can't get rid of it). If everything works, gather up all of the hadoop2-bin files from crunch-dist/target and store them in a holding area:

  cd crunch-dist/target

  mkdir /tmp/crunch-hadoop2
  cp *hadoop2-bin* /tmp/crunch-hadoop2

We'll need to go get them later when sign the artifacts. For now, close the staging repository for hadoop2 to make the release candidate available for testing and to prepare for creating the hadoop1 artifacts.

Once this process is complete, you need to reset your current branch to the last commit for the release; the same commit that you started the release against above, and then re-run the release process for hadoop1, ensuring that the release version is X.Y.Z, and that we create the apache-crunch-X.Y.Z release tag:

  git reset --hard <commit>
  mvn release:clean release:prepare
  mvn release:perform -DlocalCheckout=true

This process will also create a staging repository for the hadoop1 artifacts, once again, you should close this staging repository to make the release candidate available for testing. You can now push the tag you created against the source files to git:

  git push -u --tags origin apache-crunch-X.Y.Z

Create MD5 and SHA1 checksums for the source distribution and binary artifacts (including the hadoop2 artifacts we created earlier):

  cd crunch-dist/target
  cp /tmp/crunch-hadoop2/* .
  md5sum -b apache-crunch-X.Y.Z-src.zip > apache-crunch-X.Y.Z-src.zip.md5
  md5sum -b apache-crunch-X.Y.Z-src.tar.gz > apache-crunch-X.Y.Z-src.tar.gz.md5
  md5sum -b apache-crunch-X.Y.Z-bin.zip > apache-crunch-X.Y.Z-bin.zip.md5
  md5sum -b apache-crunch-X.Y.Z-bin.tar.gz > apache-crunch-X.Y.Z-bin.tar.gz.md5
  md5sum -b apache-crunch-X.Y.Z-hadoop2-bin.zip > apache-crunch-X.Y.Z-hadoop2-bin.zip.md5
  md5sum -b apache-crunch-X.Y.Z-hadoop2-bin.tar.gz > apache-crunch-X.Y.Z-hadoop2-bin.tar.gz.md5
  sha1sum -b apache-crunch-X.Y.Z-src.tar.gz > apache-crunch-X.Y.Z-src.tar.gz.sha1
  sha1sum -b apache-crunch-X.Y.Z-src.zip > apache-crunch-X.Y.Z-src.zip.sha1
  sha1sum -b apache-crunch-X.Y.Z-bin.zip > apache-crunch-X.Y.Z-bin.zip.sha1
  sha1sum -b apache-crunch-X.Y.Z-bin.tar.gz > apache-crunch-X.Y.Z-bin.tar.gz.sha1

  sha1sum -b apache-crunch-X.Y.Z-hadoop2-bin.zip > apache-crunch-X.Y.Z-hadoop2-bin.zip.sha1
  sha1sum -b apache-crunch-X.Y.Z-hadoop2-bin.tar.gz > apache-crunch-X.Y.Z-hadoop2-bin.tar.gz.sha1

Then copy the source distribution, the checksum file, and the detached signature to your personal web space at people.apache.org.

If you need to create a followup release candidate based on feedback on the first one, you should delete the release tag (both locally and on the central git repository) and reset your local git branch to the commit before the release preparation began, which should be two commits prior to the current HEAD.

  git tag -d apache-crunch-X.Y.Z
  git push origin :refs/tags/apache-crunch-X.Y.Z
  git reset --hard <commit>

Then perform any additional fixes or modifications that are needed based on the feedback on the release, and then restart the release process using the Maven archetypes.

Run the Vote

As per Apache policy, at least three PMC members must approve the release. There have to be more positive than negative votes; vetos are not permitted. Voting takes place on crunch-dev.

If the vote fails:

• Remove the release candidate from your web space
• Drop the staging repository
• Delete the tag
• Start over

Publish the Release

Commit the following files to Crunch's release svn repository https://dist.apache.org/repos/dist/release/crunch/:

• Source and distributions
• Checksum files
• Detached PGP signature

After a little while, the files will appear automatically in Crunch's distribution directory. Mirrors will pick them up automatically which may take up to 24 hours.

To publish the artifacts on Apache's Nexus instance, promote the build to the release repository.

Transition all resolved issues to Closed (use the batch mode for this and don't send mail for each issue) and mark the version as released (click "Administer Project").

Announce the Release

Wait for at least 24 hours to give distribution mirrors time to catch up. Update the download links on the website and add Javadocs for the release. Please use the javadoc tool from Java SE 7u25 or later due to a frame injection vulernability.

  mvn javadoc:aggregate
  cp -r target/site/apidocs /path/to/site/content/apidocs/X.Y.Z

Don't forget to update the site menu in lib/path.pm.

Send an announcement mail to users@crunch.a.o.

Clean up

Remove all previous releases from the svn distribution directory. Except for the 24 hours after the publishing a new release, the directory should only contain one release at a time. All releases are archived automatically so nothing is lost.

The information in this release guide is based on the following documents:

• Publishing Releases
• Releases FAQ
• Incubator Release Guidelines
• Publishing Maven Artifacts