As you go through the steps in this guide to perform a release, if you find anything out of date, please update it so that it is accurate for the next release.
Placeholder Values in this Document
This guide is written to be generic for any release. As such, where string literals, code snippets, or parameter/argument values are used, placeholders will be used for values that are specific for each release.
Placeholder | Description | Example Value |
---|---|---|
${BRANCH} | The branch from which the release will be made | master |
${VERSION} | The version being released (typically based off dropping the "-SNAPSHOT" suffix for the current development branch that is being released to mark a stable version | 0.3.0 |
${NEXT_VERSION} | The next development version that will being after the release is completed. Typically the next minor version with the "-SNAPSHOT" suffix. | 0.4.0-SNAPSHOT |
${JIRA_TICKET} | The ticket for the Release Manger's activities (e.g., labeling release commits, etc.) | NIFIREG-176 |
${JIRA_VERSION_URL} | The link to the version in Jira that corresponds to this release and contains all the tickets | https://issues.apache.org/jira/projects/NIFIREG/versions/12343483 |
${JIRA_RELEASE_NOTES_URL} | The link to the JIRA auto-generated release notes based on the Jira version | https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320920&version=12343483 |
${RC} | The number of the release candidate, an incrementing integer starting at 1 | 1 |
${RC_TAG_COMMIT_ID} | The 40 byte commit ID of the RC tag created during the Maven release process | 2fef9fac2b627ee1f3428b07019b333c68f65f2d |
${STAGING_REPO_ID} | The temporary Apache Nexus repository ID where staged artifacts have been placed | orgapachenifi-1132 |
| The release manager's apache user id | kdoran |
${RELEASE_TAG} | The git tag that will added to mark the final release | rel/nifi-registry-0.3.0 |
${VOTE_THREAD_URL} | The URL for the Apache Pony Mail archive of the release vote thread | https://lists.apache.org/thread.html/ca1dcd...@%3Cdev.nifi.apache.org%3E |
Prepare and Stage the Release
- Ensure you have completed all the steps in the following sections below:
- Appendix A: First Time Release Manager Prep
- Appendix B: Configure GPG, Maven, and Git for Signing and Deploying
- Create a Jira ticket for the release tasks for this release version. The resulting Jira ticket number is referred to as
${JIRA_TICKET}
in this guide. Example:
- NIFIREG-87Getting issue details... STATUS - If it does not already exist, create the
${VERSION}
in Jira from the NiFi Registry Versions management page (requires PMC group access). - Create meaningful release notes for this version if not already created. A good starting point is reviewing the Jira generated release notes from the NiFi Registry Versions management page or this JQL filter:
project = NIFIREG and fixVersion = ${VERSION}
Enter the release notes at the top of the Release Notes page of the NiFi Registry wiki: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes
It will be necessary to use a placeholder for the release date, which will be updated once the release is finalized. Create a new branch off
${BRANCH}
(typicallymaster
) named after the Jira ticket.git checkout -b ${JIRA_TICKET}-RC${RC} ${BRANCH}
Verify that Maven has sufficient heap space to perform the build tasks. Some plugins and parts of the build consume a surprisingly large amount of space. These settings have been shown to work for Java 8 for NiFi Registry version 0.2.0 and later:
export MAVEN_OPTS="-Xms1024m -Xmx3076m"
Ensure the full application builds and all tests work by executing a full build with tests. The Maven command we use for a full build is in the
.travis.yml
file at the root of the git repository. At the time of this writing, it is:mvn clean install -Pcontrib-check,integration-tests,jsUnitTests
Start and test the application from the root source folder. NiFi Registry should be up and running at
http://localhost:18080/nifi-registry
cd nifi-registry-assembly/target/nifi-registry-${VERSION}-bin/nifi-registry-${VERSION} ./bin/nifi-registry.sh start
Start and test NiFi pointing to the NiFi Registry by performing basic flow versioning operations.
- Evaluate and ensure the appropriate license headers are present on all source files.
Ensure LICENSE and NOTICE files are complete and accurate for both the source and the assembly. (Developers should always be keeping these up to date as they go along adding source and modifying dependencies to keep this burden manageable.)
Perform the Release
- Ensure you have completed all the steps in the section below: Appendix B: Configure GPG, Maven, and Git for Signing and Deploying. Otherwise, the Maven release plugin may fail at the signing or deploying stage.
Use the Maven Release Plugin to prepare the release with this command:
mvn --batch-mode release:prepare \ -Psigned_release \ -DscmCommentPrefix="${JIRA_TICKET}-RC${RC} " \ -Dtag="nifi-registry-${VERSION}-RC${RC}" \ -DreleaseVersion="${VERSION}" \ -DdevelopmentVersion="${NEXT_VERSION}" \ -Darguments="-DskipTests"
Review the release preparation results.
If problems are found,mvn release:rollback
will revert the changes, or it may also be necessary to runmvn release:clean
to get the project to a state where it can be rebuilt.If the release preparation completed without problems, perform the release and deploy artifacts to staging:
mvn release:perform \ -Psigned_release \ -DscmCommentPrefix="${JIRA_TICKET}-RC${RC} " \ -Darguments="-DskipTests"
When this completes the artifacts have been released to the Apache Nexus staging repository at https://repository.apache.org.
A local release branch has been created and there should the staging repository ID returned in a log entry like this:[INFO] * Closing staging repository with ID "orgapachenifi-1088"
This staging repository ID is referred to by
${STAGING_REPO_ID}
in this release guide.Browse to the Apache Staging Repository (https://repository.apache.org) and login with your Apache committer credentials. You should see the newly created staging repository listed. If you click on that you can inspect the various staged artifacts.
Validate that all the various aspects of the staged artifacts appear correct
Download the sources. Do they compile cleanly? If the result is a build does it execute?
- Validate the hashes match.
- Validate that the sources contain no unexpected binaries.
- Validate the signature for the build and hashes.
- Validate the LICENSE/NOTICE/Headers.
- Validate that the README is present and provides sufficient information to build and if necessary execute.
If the validated artifacts all look good then push the local git branch to the ASF repository.
git push asf ${JIRA_TICKET}-RC${RC}
From this branch, the
${RC_TAG_COMMIT_ID}
will be the 40 byte commit hash with the comment:${JIRA_TICKET}-RC${RC} prepare release nifi-registry-${VERSION}-RC${RC}
Create the signature and hashes for the source release and convenience binary files.
- ASCII armored GPG signatures (
--digest-algo=SHA512
selects the SHA512 hash algorithm). Or you can configure GPG to always prefer stronger hashes.
- Generate SHA256 hash summaries.
- Generate SHA512 hash summaries.# For the source release artifact downloaded from the staging repository, i.e. nifi-registry-${VERSION}-source-release.zip: gpg -a -b --digest-algo=SHA512 nifi-registry-${VERSION}-source-release.zip shasum -a 256 nifi-registry-${VERSION}-source-release.zip | cut -d" " -f1 > nifi-registry-${VERSION}-source-release.zip.sha256 shasum -a 512 nifi-registry-${VERSION}-source-release.zip | cut -d" " -f1 > nifi-registry-${VERSION}-source-release.zip.sha512 # For the convenience binary nifi-registry-${VERSION}-bin.zip, generated by building the downloaded source in the previous step gpg -a -b --digest-algo=SHA512 nifi-registry-${VERSION}-bin.zip shasum -a 256 nifi-registry-${VERSION}-bin.zip | cut -d" " -f1 > nifi-registry-${VERSION}-bin.zip.sha256 shasum -a 512 nifi-registry-${VERSION}-bin.zip | cut -d" " -f1 > nifi-registry-${VERSION}-bin.zip.sha512 # For the convenience binary nifi-registry-${VERSION}-bin.tar.gz, generated by building the downloaded source in the previous step gpg -a -b --digest-algo=SHA512 nifi-registry-${VERSION}-bin.tar.gz shasum -a 256 nifi-registry-${VERSION}-bin.tar.gz | cut -d" " -f1 > nifi-registry-${VERSION}-bin.tar.gz.sha256 shasum -a 512 nifi-registry-${VERSION}-bin.tar.gz | cut -d" " -f1 > nifi-registry-${VERSION}-bin.tar.gz.sha512
For reviewing of the release candidate, commit the source release and convenience binaries files along with their hashes and signatures to
https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-${VERSION}
.svn checkout https://dist.apache.org/repos/dist/dev/nifi dist-dev-nifi cd dist-dev-nifi/nifi-registry mkdir nifi-registry-${VERSION} # Add source release and binaries to nifi-registry-${VERSION}, along with their corresponding signature and hash files svn update svn add nifi-registry-${VERSION} svn commit -m "${JIRA_TICKET} Staging artifacts for nifi-registry-${VERSION}-RC${RC}" nifi-registry-${VERSION}
Error Recovery
If anything isn't correct about the staged artifacts you can drop the staged repo from repository.apache.org
and delete the local tag in git. If you also delete the local branch and clear your local maven repository under org/apache/nifi
then it is as if the release never happened. Try to figure out what went wrong so this Release Guide can be updated or corrected if necessary.
So, as has been described here, you can test the release process until you get it right. The mvn versions:set
and mvn versions:commit
commands can come in handy to help do this so you can set versions to something clearly release test related.
The Release Vote
After the release source and artifacts are staged in the repositories, the RM sends a release vote to the community.
Once the release vote is called for, members of the developer community have 72 hours to evaluate the RC and cast their vote by replying to the "[VOTE] Release ..." email sent by the RM.
NOTE: The release vote is majority rule vote that must include at least 3 binding +1 votes Apache NiFi PMC members and more positive than negative binding votes.
- RM sends a vote request email to the NiFi Developers Mailing List.
TO: dev@nifi.apache.org
FROM: ${RM_USERID}@apache.org
SUBJECT: [VOTE] Release Apache NiFi Registry ${VERSION}
BODY: RM sends the following helper email to the NiFi Developers Mailing List.
TO: dev@nifi.apache.org
FROM: ${RM_USERID}@apache.org
SUBJECT: Apache NiFi Registry ${VERSION} RC${RC} Release Helper Guide
BODY:- Developers in the community review the release candidate and reply to the vote email with their vote.
After 72 hours if at least 3 binding (PMC members) cast +1 votes, and the positive binding votes out number any negative binding votes then the vote passes and the release candidate is officially released. If the vote does not pass, corrections are made on the release branch and a new release candidate is put forward for a new vote.
RM sends vote result email.
TO: dev@nifi.apache.org
FROM: ${RM_USER}@apache.org
SUBJECT: [RESULT][VOTE] Release Apache NiFi Registry ${VERSION}
BODY:
Finalize the Release
After the vote is complete and the release is approved, these steps complete the release process.
Move convenience binaries and related artifacts from dist/dev to dist/release (note, the release branch requires PMC access):
svn move -m "${JIRA_TICKET}" https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-${VERSION} https://dist.apache.org/repos/dist/release/nifi/nifi-registry/nifi-registry-${VERSION}
- At https://repository.apache.org, login with your Apache ID credentials, go to Staging Repositories, select Release and follow the instructions on the site.
For the source code git repository, merge the release branch into master:
git checkout master git merge --no-ff ${JIRA_TICKET}-RC${RC} git push asf master
Update the Migration Guidance on the Wiki:
https://cwiki.apache.org/confluence/display/NIFI/Migration+GuidanceUpdate the wiki release notes with the final date of the release:
https://cwiki.apache.org/confluence/display/NIFI/Release+NotesUpdate the NiFi website to point to the new download(s).
- Remove older release artifacts from download page (leave the current release and the previous one).
- For the release just previous to this new one change the links to point to the archive location. See current page as an example of the needed URL changes.
- In addition to updating the download page as described delete artifacts other than the current/new release from the dist/nifi SVN storage. They are already in the archive location so no need to do anything else.If the release is on the latest development line, update the NiFi website documentation pages for NiFi Registry to match the release.
- Run the NiFi Registry ${VERSION}
- Pull down the documentation by runningwget -prk http://${host}:${port}/nifi-registry-docs/documentation
- Rename the file index file that was generated by runningmv ${host}:${port}/nifi-registry-docs/documentation ${host}:${port}/nifi-registry-docs/index.html
- Use svn to replace the nifi and nifi-docs folders inhttps://svn.apache.org/repos/asf/nifi/site/trunk/docs with those in the ${host}:${port}
directory pulled down aboveIn Jira, mark the ${JIRA_TICKET} resolved. All tickets for the Jira release version should now be resolved or closed.
Mark the release version for ${VERSION} as 'Released' using the NiFi Registry Versions management page.
This may also be a good time to create a Jira version for the next planned release of ${NEXT_VERSION} (usually the next minor version).Ensure the release artifacts are successfully mirrored to the archive, specifically
https://archive.apache.org/dist/nifi/nifi-registry/nifi-registry-${VERSION}/nifi-registry-${VERSION}-bin.tar.gz.
If there were any issues with the above listed file not being available, it may be necessary to reach out to the ASF Infra team.Docker Image Note
When the Docker image build is automated (as of this writing, not currently setup for NiFi Registry), then it is triggered by pushing the signed tag in the next step. The release artifacts must be present in the archive before continuing.Create a proper signed tag of the released codebase based on the RC tag created during the Maven release process.
For instructions on setting up to sign your tag see: http://gitready.com/advanced/2014/11/02/gpg-sign-releases.html or the Appendix B section below.# create the tag in your local repo git tag -s rel/nifi-registry-${VERSION} -m "${JIRA_TICKET} signed release tag for approved release of NiFi Registry ${VERSION}" ${RC_TAG_COMMIT_ID} # push the tag to the ASF repo git push asf rel/nifi-registry-${VERSION}
- Build the new Docker image based on the artifacts accessible via the archive.apache.org mirror. Tag the image as the current version and latest and push to the official repository. (TODO, expand these instructions for doing this manually or setup automation with ASF Infra as we have to NiFi.)
Update Docker version information for the dockerhub image to point to the next release. For instance, if the ${NEXT_VERSION} is 0.4.0-SNAPSHOT, these values should be updated to 0.4.0, so that at the time of the next release they will reference the correct version number. This currently consists of two files:
-
nifi-registry-core/nifi-registry-docker/dockerhub/Dockerfile
, Line 24, and
-nifi-registry-core/nifi-registry-docker/dockerhub/DockerImage.txt
, Line 16.After the release has been complete for 24 hours send the release announcement email.
For an understanding of why you need to wait 24 hours, see: https://www.apache.org/dev/release.html#release-announcementsTO: announce@apache.org, dev@nifi.apache.org
REPLY-TO: dev@nifi.apache.org
FROM: ${RM_USER}@apache.org
SUBJECT: [ANNOUNCE] Apache NiFi Registry ${VERSION} ReleaseBODY:
Appendix A: First Time Release Manager Prep
These steps only have to be done once for any NiFi project release (NiFi, NiFi Registry, NiFi MiNiFi, etc.).
- Make sure you can login (using your Apache ID LDAP credentials) the Apache Nexus Server at:
https://repository.apache.org Make sure you have write access to the Apache NiFi Wiki (for publishing release notes):
https://cwiki.apache.org/confluence/display/NIFI/Release+NotesMake sure your public PGP key has been added to: https://dist.apache.org/repos/dist/release/nifi/KEYS
If you have never signed a release before, recommended to read http://www.apache.org/dev/release-signing.html.
If you don't have a code signing key already, follow the instructions there to generate and publish a new key pair.
Once you have a key, add the public key to the dev KEYS file in the SVN repository.
If you use SVN infrequently, you may find this a useful reference for the command syntax: https://www.cheatography.com/davechild/cheat-sheets/subversion/# Checkout the SVN repo: svn checkout https://dist.apache.org/repos/dist/dev/nifi dist-dev-nifi cd dist-dev-nifi # Follow the instructions at the top of the KEYS file to append your public code signing key gpg --list-sigs --fingerprint ${NAME_OR_KEYID} >> KEYS gpg --armor --export ${NAME_OR_KEYID} >> KEYS # Verify your key information has been appended correctly to the KEYS file # Commit the file back to the SVN repo svn commit KEYS -m "Adding code signing key to KEYS file" # Prior to the release, move the KEYS file to release, as per https://www.apache.org/dev/openpgp.html#update-KEYS # (A PMC member will need to do this) svn checkout https://dist.apache.org/repos/dist/release/nifi dist-release-nifi
- Make sure your maven settings.xml and settings-security.xml files are configured correctly to sign build artifacts, as described in Appendix B: Configure GPG, Maven, and Git for Signing and Deploying.
Appendix B: Configure GPG, Maven, and Git for Signing and Deploying
This sections assumes that you have already completed the steps in Appendix A: First Time Release Manager Prep in order to generate a code signing key pair. Generating the signing keys is a one time step; so long as your key is not comprised or otherwise invalid it can be used for multiple releases. The following settings need to be configured before every release (or at least checked that they do not need to be changed) in order to make sure that the steps that perform a PGP signature will work correctly.
GPG Settings
If you use a different GPG home directory for code signing, enable that:
export GNUPGHOME=/path/tp/code-signing-gnupg-home # Note: # - Verify this setting by running `gpg --version` # - Confirm your code signing key by running `gpg --list-keys <email_address>` # - To clear this env var and revert this, use `unset GNUPGHOME`, or just terminate the shell session and create a new one.
(Recommended) Configure GPG to always prefer stronger hashes: https://www.apache.org/dev/openpgp.html#key-gen-avoid-sha1
Note that you may have to configure this in order for Maven to prompt you for your private key passphrase:
GPG_TTY=$(tty) && export GPG_TTY
Maven Settings
To perform a release, your Maven settings.xml
, typically located in your ~/.m2
directory, must be updated to include a signed_release
profile and a server
entry for repository.apache.org:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd"> <profiles> <profile> <id>signed_release</id> <properties> <mavenExecutorId>forked-path</mavenExecutorId> <gpg.keyname>${RM_USER}@apache.org</gpg.keyname> <gpg.passphrase>ENCRYPTED_GPG_PASSPHRASE_HERE</gpg.passphrase> </properties> </profile> </profiles> <servers> <server> <id>repository.apache.org</id> <username>${RM_USER}</username> <password>ENCRYPTED_PASSWORD_HERE</password> </server> </servers> </settings>
Encrypting Passwords in Maven settings.xml
Steps to configure and encrypt Maven passwords are available at:
https://blog.sonatype.com/2009/10/maven-tips-and-tricks-encrypting-passwords/
There are other ways to ensure your PGP key is available for signing as well.
Git Settings
During the RM process, git will use your code signing private key in order to sign commits and tags. To facilitate this, configure the git user.signingkey
property, such as:
# Check the current value (you may want to make a note of it so that you can revert this setting after you are done the release): git config --global user.signingkey # Set a temporary value for your code signing key: git config --global user.signingkey CODE_SIGNING_KEY_FINGERPRINT # Discover your CODE_SIGNING_KEY_FINGERPRINT by running `gpg --list-keys <email_address>`