As you go through the steps in this guide to perform a release, if you find anything out of date, please update it so that it is accurate for the next release.
Placeholder Values in this Document
This guide is written to be generic for any release. As such, where string literals, code snippets, or parameter/argument values are used, placeholders will be used for values that are specific for each release.
Placeholder | Description | Example Value |
---|---|---|
${BRANCH} | The branch from which the release will be made | master |
${VERSION} | The version being released (typically based off dropping the "-SNAPSHOT" suffix for the current development branch that is being released to mark a stable version | 0.3.0 |
${NEXT_VERSION} | The next development version that will being after the release is completed. Typically the next minor version with the "-SNAPSHOT" suffix. | 0.4.0-SNAPSHOT |
${JIRA_TICKET} | The ticket for the Release Manger's activities (e.g., labeling release commits, etc.) | NIFIREG-176 |
${JIRA_VERSION_URL} | The link to the version in Jira that corresponds to this release and contains all the tickets | https://issues.apache.org/jira/projects/NIFIREG/versions/12343483 |
${JIRA_RELEASE_NOTES_URL} | The link to the JIRA auto-generated release notes based on the Jira version | https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320920&version=12343483 |
${RC} | The number of the release candidate, an incrementing integer starting at 1 | 1 |
${RC_TAG_COMMIT_ID} | The 40 byte commit ID of the RC tag created during the Maven release process | 2fef9fac2b627ee1f3428b07019b333c68f65f2d |
${STAGING_REPO_ID} | The temporary Apache Nexus repository ID where staged artifacts have been placed | orgapachenifi-1132 |
| The release manager's apache user id | kdoran |
${RELEASE_TAG} | The git tag that will added to mark the final release | rel/nifi-registry-0.3.0 |
${VOTE_THREAD_URL} | The URL for the Apache Pony Mail archive of the release vote thread | https://lists.apache.org/thread.html/ca1dcd5a30a2d21a49c490fd59b44e2c01c22f2e2bb15ba0fc1904cb@%3Cdev.nifi.apache.org%3E |
Prepare and Stage the Release
- Ensure you have completed all the steps in the following sections below:
- Appendix A: First Time Release Manager Prep
- Appendix B: Configure GPG, Maven, and Git for Signing and Deploying
- Create a Jira ticket for the release tasks for this release version. The resulting Jira ticket number is referred to as
${JIRA_TICKET}
in this guide. Example:
- NIFIREG-87Getting issue details... STATUS - If it does not already exist, create the
${VERSION}
in Jira from the NiFi Registry Versions management page (requires PMC group access). - Create meaningful release notes for this version if not already created. A good starting point is reviewing the Jira generated release notes from the NiFi Registry Versions management page or this JQL filter:
project = NIFIREG and fixVersion = ${VERSION}
Enter the release notes at the top of the Release Notes page of the NiFi Registry wiki: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes
It will be necessary to use a placeholder for the release date, which will be updated once the release is finalized. Create a new branch off
${BRANCH}
(typicallymaster
) named after the Jira ticket.git checkout -b ${JIRA_TICKET}-RC${RC} ${BRANCH}
Verify that Maven has sufficient heap space to perform the build tasks. Some plugins and parts of the build consume a surprisingly large amount of space. These settings have been shown to work for Java 8 for NiFi Registry version 0.2.0 and later:
export MAVEN_OPTS="-Xms1024m -Xmx3076m"
Ensure the full application builds and all tests work by executing a full build with tests. The Maven command we use for a full build is in the
.travis.yml
file at the root of the git repository. At the time of this writing, it is:mvn clean install -Pcontrib-check,integration-tests,jsUnitTests
Start and test the application from the root source folder. NiFi Registry should be up and running at
http://localhost:18080/nifi-registry
cd nifi-registry-assembly/target/nifi-registry-${VERSION}-bin/nifi-registry-${VERSION} ./bin/nifi-registry.sh start
Start and test NiFi pointing to the NiFi Registry by performing basic flow versioning operations.
- Evaluate and ensure the appropriate license headers are present on all source files.
Ensure LICENSE and NOTICE files are complete and accurate for both the source and the assembly. (Developers should always be keeping these up to date as they go along adding source and modifying dependencies to keep this burden manageable.)
Perform the Release
- Ensure you have completed all the steps in the section below: Appendix B: Configure GPG, Maven, and Git for Signing and Deploying. Otherwise, the Maven release plugin may fail at the signing or deploying stage.
Use the Maven Release Plugin to prepare the release with this command:
mvn --batch-mode release:prepare \ -Psigned_release \ -DscmCommentPrefix="${JIRA_TICKET}-RC${RC} " \ -Dtag="nifi-registry-${VERSION}-RC${RC}" \ -DreleaseVersion="${VERSION}" \ -DdevelopmentVersion="${NEXT_VERSION}" \ -Darguments="-DskipTests"
Review the release preparation results.
If problems are found,mvn release:rollback
will revert the changes, or it may also be necessary to runmvn release:clean
to get the project to a state where it can be rebuilt.If the release preparation completed without problems, perform the release and deploy artifacts to staging:
mvn release:perform \ -Psigned_release \ -DscmCommentPrefix="${JIRA_TICKET}-RC${RC} " \ -Darguments="-DskipTests"
When this completes the artifacts have been released to the Apache Nexus staging repository at https://repository.apache.org.
A local release branch has been created and there should the staging repository ID returned in a log entry like this:[INFO] * Closing staging repository with ID "orgapachenifi-1088"
This staging repository ID is referred to by
${STAGING_REPO_ID}
in this release guide.Browse to the Apache Staging Repository (https://repository.apache.org) and login with your Apache committer credentials. You should see the newly created staging repository listed. If you click on that you can inspect the various staged artifacts.
Validate that all the various aspects of the staged artifacts appear correct
Download the sources. Do they compile cleanly? If the result is a build does it execute?
- Validate the hashes match.
- Validate that the sources contain no unexpected binaries.
- Validate the signature for the build and hashes.
- Validate the LICENSE/NOTICE/Headers.
- Validate that the README is present and provides sufficient information to build and if necessary execute.
If the validated artifacts all look good then push the local git branch to the ASF repository.
git push asf ${JIRA_TICKET}-RC${RC}
From this branch, the
${RC_TAG_COMMIT_ID}
will be the 40 byte commit hash with the comment:${JIRA_TICKET}-RC${RC} prepare release nifi-registry-${VERSION}-RC${RC}
Create the signature and hashes for the source release and convenience binary files.
- ASCII armored GPG signatures (--digest-algo=SHA512 selects the SHA512 hash algorithm). Or you can configure GPG to always prefer stronger hashes.
- Generate SHA256 hash summaries.
- Generate SHA512 hash summaries.# For the source release artifact downloaded from the staging repository, i.e. nifi-registry-${VERSION}-source-release.zip: gpg -a -b --digest-algo=SHA512 nifi-registry-${VERSION}-source-release.zip shasum -a 256 nifi-registry-${VERSION}-source-release.zip | cut -d" " -f1 > nifi-registry-${VERSION}-source-release.zip.sha256 shasum -a 512 nifi-registry-${VERSION}-source-release.zip | cut -d" " -f1 > nifi-registry-${VERSION}-source-release.zip.sha512 # For the convenience binary (i.e. nifi-registry-${VERSION}-bin.zip, generated by building the downloaded source in the previous step) gpg -a -b --digest-algo=SHA512 nifi-registry-${VERSION}-bin.zip shasum -a 256 nifi-registry-${VERSION}-bin.zip | cut -d" " -f1 > nifi-registry-${VERSION}-bin.zip.sha256 shasum -a 512 nifi-registry-${VERSION}-bin.zip | cut -d" " -f1 > nifi-registry-${VERSION}-bin.zip.sha512 # Note, we used to also provide a .tar.gz alternative for the binary, but we are moving away from that to simplify things and cut down on upload times.
For reviewing of the release candidate, commit the source release and convenience binaries files along with their hashes and signatures to
https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-${VERSION}
.svn checkout https://dist.apache.org/repos/dist/dev/nifi dist-dev-nifi cd dist-dev-nifi/nifi-registry mkdir nifi-registry-${VERSION} # Add source release and binaries to nifi-registry-${VERSION}, along with their corresponding signature and hash files svn update svn add nifi-registry-${VERSION} svn commit -m "${JIRA_TICKET} Staging artifacts for nifi-registry-${VERSION}-RC${RC}" nifi-registry-${VERSION}
Error Recovery
If anything isn't correct about the staged artifacts you can drop the staged repo from repository.apache.org
and delete the local tag in git. If you also delete the local branch and clear your local maven repository under org/apache/nifi
then it is as if the release never happened. Try to figure out what went wrong so this Release Guide can be updated or corrected if necessary.
So, as has been described here, you can test the release process until you get it right. The mvn versions:set
and mvn versions:commit
commands can come in handy to help do this so you can set versions to something clearly release test related.
Start the Release Vote
After the release source and artifacts are staged in the repositories, the RM sends a release vote to the community.
Once the release vote is called for, members of the developer community have 72 hours to evaluate the RC and cast their vote by replying to the "[VOTE] Release ..." email sent by the RM.
NOTE: The release vote is majority rule vote that must include at least 3 binding +1 votes Apache NiFi PMC members and more positive than negative binding votes.
- RM sends a vote request email to the NiFi Developers Mailing List.
TO: dev@nifi.apache.org
FROM: ${RM_USERID}@apache.org
SUBJECT: [VOTE] Release Apache NiFi Registry ${VERSION} RM sends the following helper email to the NiFi Developers Mailing List.
TO: dev@nifi.apache.org
FROM: ${RM_USERID}@apache.org
SUBJECT: Apache NiFi Registry ${VERSION} RC${RC} Release Helper Guide
- Developers in the community review the release candidate and reply to the vote email with their vote.
After 72 hours if at least 3 binding (PMC members) cast +1 votes, and the positive binding votes out number any negative binding votes then the vote passes and the release candidate is officially released. If the vote does not pass, corrections are made on the release branch and a new release candidate is put forward for a new vote.
- RM sends vote result email.
TODO
Finalize the Release (after the vote)
Appendix A: First Time Release Manager Prep
These steps only have to be done once for any NiFi project release (NiFi, NiFi Registry, NiFi MiNiFi, etc.).
- Make sure you can login (using your Apache ID LDAP credentials) the Apache Nexus Server at:
https://repository.apache.org Make sure you have write access to the Apache NiFi Wiki (for publishing release notes):
https://cwiki.apache.org/confluence/display/NIFI/Release+NotesMake sure your public PGP key has been added to: https://dist.apache.org/repos/dist/release/nifi/KEYS
If you have never signed a release before, recommended to read http://www.apache.org/dev/release-signing.html.
If you don't have a code signing key already, follow the instructions there to generate and publish a new key pair.
Once you have a key, add the public key to the dev KEYS file in the svn repository.
# Checkout the SVN repo: svn checkout https://dist.apache.org/repos/dist/dev/nifi dist-dev-nifi cd dist-dev-nifi # Follow the instructions at the top of the KEYS file to append your public code signing key gpg --list-sigs --fingerprint ${NAME_OR_KEYID} >> KEYS gpg --armor --export ${NAME_OR_KEYID} >> KEYS # Verify your key information has been appended correctly to the KEYS file # Commit the file back to the SVN repo svn commit KEYS -m "Adding code signing key to KEYS file" # Prior to the release, move the KEYS file to release, as per https://www.apache.org/dev/openpgp.html#update-KEYS # (A PMC member will need to do this) svn checkout https://dist.apache.org/repos/dist/release/nifi dist-release-nifi
- Make sure your maven settings.xml and settings-security.xml files are configured correctly to sign build artifacts, as described in Appendix B: Configure GPG, Maven, and Git for Signing and Deploying.
Appendix B: Configure GPG, Maven, and Git for Signing and Deploying
This sections assumes that you have already completed the steps in Appendix A: First Time Release Manager Prep in order to generate a code signing key pair. Generating the signing keys is a one time step; so long as your key is not comprised or otherwise invalid it can be used for multiple releases. The following settings need to be configured before every release (or at least checked that they do not need to be changed) in order to make sure that the steps that perform a PGP signature will work correctly.
GPG Settings
If you use a different GPG home directory for code signing, enable that:
export GNUPGHOME=/path/tp/code-signing-gnupg-home # Note: # - Verify this setting by running `gpg --version` # - Confirm your code signing key by running `gpg --list-keys <email_address>` # - To clear this env var and revert this, use `unset GNUPGHOME`, or just terminate the shell session and create a new one.
(Recommended) Configure GPG to always prefer stronger hashes: https://www.apache.org/dev/openpgp.html#key-gen-avoid-sha1
Note that you may have to configure this in order for Maven to prompt you for your private key passphrase:
GPG_TTY=$(tty) && export GPG_TTY
Maven Settings
To perform a release, your Maven settings.xml
, typically located in your ~/.m2
directory, must be updated to include a signed_release
profile and a server
entry for repository.apache.org:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd"> <profiles> <profile> <id>signed_release</id> <properties> <mavenExecutorId>forked-path</mavenExecutorId> <gpg.keyname>${RM_USER}@apache.org</gpg.keyname> <gpg.passphrase>ENCRYPTED_GPG_PASSPHRASE_HERE</gpg.passphrase> </properties> </profile> </profiles> <servers> <server> <id>repository.apache.org</id> <username>${RM_USER}</username> <password>ENCRYPTED_PASSWORD_HERE</password> </server> </servers> </settings>
Encrypting Passwords in Maven settings.xml
Steps to configure and encrypt Maven passwords are available at:
https://blog.sonatype.com/2009/10/maven-tips-and-tricks-encrypting-passwords/
There are other ways to ensure your PGP key is available for signing as well.
Git Settings
During the RM process, git will use your code signing private key in order to sign commits and tags. To facilitate this, configure the git user.signingkey
property, such as:
# Check the current value (you may want to make a note of it so that you can revert this setting after you are done the release): git config --global user.signingkey # Set a temporary value for your code signing key: git config --global user.signingkey CODE_SIGNING_KEY_FINGERPRINT # Discover your CODE_SIGNING_KEY_FINGERPRINT by running `gpg --list-keys <email_address>`