| Table of Contents |
|---|
| Info |
|---|
As you go through the steps in this guide to perform a release, if you find anything out of date, please update it so that it is accurate for the next release. |
Placeholder Values in this Document
This guide is written to be generic for any release. As such, where string literals, code snippets, or parameter/argument values are used, placeholders will be used for values that are specific for each release.
...
The next development version that will being after the release is completed. Typically the next minor version with the "-SNAPSHOT" suffix.
...
${RM_USER}
...
Prepare and Stage the Release
...
| Jira | ||||||
|---|---|---|---|---|---|---|
|
...
page
...
Create a new branch off ${BRANCH} (typically master) named after the Jira ticket.
| Code Block | ||
|---|---|---|
| ||
git checkout -b ${JIRA_TICKET}-RC${RC} ${BRANCH} |
Verify that Maven has sufficient heap space to perform the build tasks. Some plugins and parts of the build consume a surprisingly large amount of space. These settings have been shown to work for Java 8 for NiFi Registry version 0.2.0 and later:
| Code Block | ||
|---|---|---|
| ||
export MAVEN_OPTS="-Xms1024m -Xmx3076m" |
Ensure the full application builds and all tests work by executing a full build with tests. The Maven command we use for a full build is in the .travis.yml file at the root of the git repository. At the time of this writing, it is:
| Code Block | ||
|---|---|---|
| ||
mvn clean install -Pcontrib-check,integration-tests,jsUnitTests |
Start and test the application from the root source folder. NiFi Registry should be up and running at http://localhost:18080/nifi-registry
| Code Block | ||
|---|---|---|
| ||
cd nifi-registry-assembly/target/nifi-registry-${VERSION}-bin/nifi-registry-${VERSION}
./bin/nifi-registry.sh start |
...
Ensure LICENSE and NOTICE files are complete and accurate for both the source and the assembly. (Developers should always be keeping these up to date as they go along adding source and modifying dependencies to keep this burden manageable.)
Perform the Release
...
Use the Maven Release Plugin to prepare the release with this command:
| Code Block | ||
|---|---|---|
| ||
mvn --batch-mode release:prepare \
-Psigned_release \
-DscmCommentPrefix="${JIRA_TICKET}-RC${RC} " \
-Dtag="nifi-registry-${VERSION}-RC${RC}" \
-DreleaseVersion="${VERSION}" \
-DdevelopmentVersion="${NEXT_VERSION}" \
-Darguments="-DskipTests" |
...
If the release preparation completed without problems, perform the release and deploy artifacts to staging:
| Code Block | ||
|---|---|---|
| ||
mvn release:perform \
-Psigned_release \
-DscmCommentPrefix="${JIRA_TICKET}-RC${RC} " \
-Darguments="-DskipTests" |
When this completes the artifacts have been released to the Apache Nexus staging repository at https://repository.apache.org.
A local release branch has been created and there should the staging repository ID returned in a log entry like this:
[INFO] * Closing staging repository with ID "orgapachenifi-1088"
...
Browse to the Apache Staging Repository (https://repository.apache.org) and login with your Apache committer credentials. You should see the newly created staging repository listed. If you click on that you can inspect the various staged artifacts.
Validate that all the various aspects of the staged artifacts appear correct
...
Download the sources. Do they compile cleanly? If the result is a build does it execute?
...
If the validated artifacts all look good then push the local git branch to the ASF repository.
| Code Block | ||
|---|---|---|
| ||
git push asf ${JIRA_TICKET}-RC${RC} |
...
Create the signature and hashes for the source release and convenience binary files.
- ASCII armored GPG signatures (--digest-algo=SHA512 selects the SHA512 hash algorithm). Or you can configure GPG to always prefer stronger hashes.
- Generate SHA256 hash summaries.
- Generate SHA512 hash summaries.
| Code Block | ||
|---|---|---|
| ||
# For the source release artifact downloaded from the staging repository, i.e. nifi-registry-${VERSION}-source-release.zip:
gpg -a -b --digest-algo=SHA512 nifi-registry-${VERSION}-source-release.zip
shasum -a 256 nifi-registry-${VERSION}-source-release.zip | cut -d" " -f1 > nifi-registry-${VERSION}-source-release.zip.sha256
shasum -a 512 nifi-registry-${VERSION}-source-release.zip | cut -d" " -f1 > nifi-registry-${VERSION}-source-release.zip.sha512
# For the convenience binary nifi-registry-${VERSION}-bin.zip, generated by building the downloaded source in the previous step
gpg -a -b --digest-algo=SHA512 nifi-registry-${VERSION}-bin.zip
shasum -a 256 nifi-registry-${VERSION}-bin.zip | cut -d" " -f1 > nifi-registry-${VERSION}-bin.zip.sha256
shasum -a 512 nifi-registry-${VERSION}-bin.zip | cut -d" " -f1 > nifi-registry-${VERSION}-bin.zip.sha512
# For the convenience binary nifi-registry-${VERSION}-bin.tar.gz, generated by building the downloaded source in the previous step
gpg -a -b --digest-algo=SHA512 nifi-registry-${VERSION}-bin.tar.gz
shasum -a 256 nifi-registry-${VERSION}-bin.tar.gz | cut -d" " -f1 > nifi-registry-${VERSION}-bin.tar.gz.sha256
shasum -a 512 nifi-registry-${VERSION}-bin.tar.gz | cut -d" " -f1 > nifi-registry-${VERSION}-bin.tar.gz.sha512 |
For reviewing of the release candidate, commit the source release and convenience binaries files along with their hashes and signatures to https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-${VERSION}.
| Code Block | ||
|---|---|---|
| ||
svn checkout https://dist.apache.org/repos/dist/dev/nifi dist-dev-nifi
cd dist-dev-nifi/nifi-registry
mkdir nifi-registry-${VERSION}
# Add source release and binaries to nifi-registry-${VERSION}, along with their corresponding signature and hash files
svn update
svn add nifi-registry-${VERSION}
svn commit -m "${JIRA_TICKET} Staging artifacts for nifi-registry-${VERSION}-RC${RC}" nifi-registry-${VERSION} |
Error Recovery
If anything isn't correct about the staged artifacts you can drop the staged repo from repository.apache.org and delete the local tag in git. If you also delete the local branch and clear your local maven repository under org/apache/nifi then it is as if the release never happened. Try to figure out what went wrong so this Release Guide can be updated or corrected if necessary.
So, as has been described here, you can test the release process until you get it right. The mvn versions:set and mvn versions:commit commands can come in handy to help do this so you can set versions to something clearly release test related.
The Release Vote
After the release source and artifacts are staged in the repositories, the RM sends a release vote to the community.
Once the release vote is called for, members of the developer community have 72 hours to evaluate the RC and cast their vote by replying to the "[VOTE] Release ..." email sent by the RM.
NOTE: The release vote is majority rule vote that must include at least 3 binding +1 votes Apache NiFi PMC members and more positive than negative binding votes.
...
TO: dev@nifi.apache.org
FROM: ${RM_USERID}@apache.org
SUBJECT: [VOTE] Release Apache NiFi Registry ${VERSION}
BODY:
...
| title | Template for the email body. Click here to expand... |
|---|
...
has moved to the NiFi Registry Space (NIFIREG): https://cwiki.apache.org/confluence/display/
...
RM sends the following helper email to the NiFi Developers Mailing List.TO: dev@nifi.apache.orgFROM: ${RM_USERID}@apache.orgSUBJECT: Apache NiFi Registry ${VERSION} RC${RC} Release Helper Guide
BODY:
...
| title | Template for the email body. Click here to expand... |
|---|
| No Format |
|---|
Hello Apache NiFi community,
Please find the associated guidance to help those interested in
validating/verifying the Apache NiFi Registry release so they can vote.
# Download latest KEYS file:
https://dist.apache.org/repos/dist/dev/nifi/KEYS
# Import keys file:
gpg --import KEYS
# [optional] Clear out local maven artifact repository
# Pull down nifi-registry-${VERSION} source release artifacts for review:
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-${VERSION}/nifi-registry-${VERSION}-source-release.zip
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-${VERSION}/nifi-registry-${VERSION}-source-release.zip.asc
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-${VERSION}/nifi-registry-${VERSION}-source-release.zip.sha256
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-${VERSION}/nifi-registry-${VERSION}-source-release.zip.sha512
# Verify the signature
gpg --verify nifi-registry-${VERSION}-source-release.zip.asc
# Verify the hashes (sha256, sha512) match the source and what was provided in the vote email thread
shasum -a 1 nifi-registry-${VERSION}-source-release.zip
shasum -a 256 nifi-registry-${VERSION}-source-release.zip
shasum -a 512 nifi-registry-${VERSION}-source-release.zip
# Unzip nifi-registry-${VERSION}-source-release.zip
# Verify the build works including release audit tool (RAT) checks
cd nifi-registry-${VERSION}
mvn clean install -Pcontrib-check
# Verify the contents contain a good README, NOTICE, and LICENSE.
# Verify the git commit ID is correct
# Verify the RC was branched off the correct git commit ID
# Look at the resulting convenience binary as found in nifi-registry-assembly/target
# Make sure the README, NOTICE, and LICENSE are present and correct
# Run the resulting convenience binary and make sure it works as expected
# Test integration between the Registry and NiFi
Start the registry
./bin/nifi-registry.sh start
Create a bucket in the registry
- Go to the registry UI at http://localhost:18080/nifi-registry
- Click the tool icon in the top right corner
- Click New Bucket from the bucket table
- Enter a name and click create
Start NiFi
Tell NiFi about your local registry instance
- Go the controller settings for NiFi from the top-right menu
- Select the Registry Clients tab
- Add a new Registry Client giving it a name and the url of http://localhost:18080
Create a process group and place it under version control
- Right click on the PG and select the Version menu
- Select Start Version Control
- Choose the registry instance and bucket you want to use
- Enter a name, description, and comment
Go back to the registry and refresh the main page and you should see the versioned flow you just saved
Import a new PG from a versioned flow
- Drag on a new PG like normal
- Instead of entering a name, click the Import link
- Now choose the flow you saved before
You should have a second identical PG now.
# Send a response to the vote thread indicating a +1, 0, -1 based on your findings.
Thank you for your time and effort to validate the release! |
...
RM sends vote result email.TO: dev@nifi.apache.org
FROM: ${RM_USER}@apache.org
SUBJECT: [RESULT][VOTE] Release Apache NiFi Registry ${VERSION}
BODY:
...
| title | Template for email body. Click here to expand... |
|---|
| No Format |
|---|
Apache NiFi Community,
I am pleased to announce that the nifi-registry-${VERSION} release of Apache NiFi Registry passes with
X +1 (binding) votes
Y +1 (non-binding) votes
0 0 votes
0 -1 votes
Thanks to all who helped make this release possible.
Here is the vote thread:
${VOTE_THREAD_URL} |
Finalize the Release
After the vote is complete and the release is approved, these steps complete the release process.
Move convenience binaries and related artifacts from dist/dev to dist/release (note, the release branch requires PMC access):
| Code Block | ||
|---|---|---|
| ||
svn move -m "${JIRA_TICKET}" https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-${VERSION} https://dist.apache.org/repos/dist/release/nifi/nifi-registry/nifi-registry-${VERSION} |
...
For the source code git repository, merge the release branch into master:
| Code Block | ||
|---|---|---|
| ||
git checkout master
git merge --no-ff ${JIRA_TICKET}-RC${RC}
git push asf master |
...
Ensure the release artifacts are successfully mirrored to the archive, specifically https://archive.apache.org/dist/nifi/nifi-registry/nifi-registry-${VERSION}/nifi-registry-${VERSION}-bin.tar.gz.
If there were any issues with the above listed file not being available, it may be necessary to reach out to the ASF Infra team.
| Info | ||
|---|---|---|
| ||
| When the Docker image build is automated (as of this writing, not currently setup for NiFi Registry), then it is triggered by pushing the signed tag in the next step. The release artifacts must be present in the archive before continuing. |
Create a proper signed tag of the released codebase based on the RC tag created during the Maven release process.
For instructions on setting up to sign your tag see: http://gitready.com/advanced/2014/11/02/gpg-sign-releases.html or the Appendix B section below.
| Code Block | ||
|---|---|---|
| ||
# create the tag in your local repo
git tag -s rel/nifi-registry-${VERSION} -m "${JIRA_TICKET} signed release tag for approved release of NiFi Registry ${VERSION}" ${RC_TAG_COMMIT_ID}
# push the tag to the ASF repo
git push asf rel/nifi-registry-${VERSION} |
...
Update Docker version information for the dockerhub image to point to the next release. For instance, if the ${NEXT_VERSION} is 0.4.0-SNAPSHOT, these values should be updated to 0.4.0, so that at the time of the next release they will reference the correct version number. This currently consists of two files:
...
After the release has been complete for 24 hours send the release announcement email.
| Info |
|---|
| For an understanding of why you need to wait 24 hours, see: https://www.apache.org/dev/release.html#release-announcements |
...
| title | Template for email body. Click here to expand... |
|---|
| No Format |
|---|
Hello,
The Apache NiFi team would like to announce the release of Apache NiFi Registry ${VERSION}.
Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for dataflow. It supports highly configurable directed graphs of data routing, transformation, and system mediation logic.
Registry, a subproject of Apache NiFi, is a complementary application that provides a central location for storage and management of shared resources across one or more instances of NiFi or MiNiFi.
More details on Apache NiFi Registry, along with release artifacts, can be found here:
https://nifi.apache.org/registry.html
Maven artifacts have been made available here:
https://repository.apache.org/content/repositories/releases/org/apache/nifi/registry/
Issues closed/resolved for this list can be found here:
${JIRA_RELEASE_NOTES_URL}
Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiRegistry${VERSION}
Thank you,
The Apache NiFi team |
Appendix A: First Time Release Manager Prep
These steps only have to be done once for any NiFi project release (NiFi, NiFi Registry, NiFi MiNiFi, etc.).
...
If you have never signed a release before, recommended to read http://www.apache.org/dev/release-signing.html.
...
If you don't have a code signing key already, follow the instructions there to generate and publish a new key pair.
Once you have a key, add the public key to the dev KEYS file in the SVN repository.
If you use SVN infrequently, you may find this a useful reference for the command syntax: https://www.cheatography.com/davechild/cheat-sheets/subversion/
| Code Block | ||
|---|---|---|
| ||
# Checkout the SVN repo:
svn checkout https://dist.apache.org/repos/dist/dev/nifi dist-dev-nifi
cd dist-dev-nifi
# Follow the instructions at the top of the KEYS file to append your public code signing key
gpg --list-sigs --fingerprint ${NAME_OR_KEYID} >> KEYS
gpg --armor --export ${NAME_OR_KEYID} >> KEYS
# Verify your key information has been appended correctly to the KEYS file
# Commit the file back to the SVN repo
svn commit KEYS -m "Adding code signing key to KEYS file"
# Prior to the release, move the KEYS file to release, as per https://www.apache.org/dev/openpgp.html#update-KEYS
# (A PMC member will need to do this)
svn checkout https://dist.apache.org/repos/dist/release/nifi dist-release-nifi |
...
Appendix B: Configure GPG, Maven, and Git for Signing and Deploying
This sections assumes that you have already completed the steps in Appendix A: First Time Release Manager Prep in order to generate a code signing key pair. Generating the signing keys is a one time step; so long as your key is not comprised or otherwise invalid it can be used for multiple releases. The following settings need to be configured before every release (or at least checked that they do not need to be changed) in order to make sure that the steps that perform a PGP signature will work correctly.
GPG Settings
If you use a different GPG home directory for code signing, enable that:
| Code Block | ||
|---|---|---|
| ||
export GNUPGHOME=/path/tp/code-signing-gnupg-home
# Note:
# - Verify this setting by running `gpg --version`
# - Confirm your code signing key by running `gpg --list-keys <email_address>`
# - To clear this env var and revert this, use `unset GNUPGHOME`, or just terminate the shell session and create a new one. |
(Recommended) Configure GPG to always prefer stronger hashes: https://www.apache.org/dev/openpgp.html#key-gen-avoid-sha1
Note that you may have to configure this in order for Maven to prompt you for your private key passphrase:
| Code Block | ||
|---|---|---|
| ||
GPG_TTY=$(tty) && export GPG_TTY |
Maven Settings
To perform a release, your Maven settings.xml, typically located in your ~/.m2 directory, must be updated to include a signed_release profile and a server entry for repository.apache.org:
| Code Block | ||
|---|---|---|
| ||
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
<profiles>
<profile>
<id>signed_release</id>
<properties>
<mavenExecutorId>forked-path</mavenExecutorId>
<gpg.keyname>${RM_USER}@apache.org</gpg.keyname>
<gpg.passphrase>ENCRYPTED_GPG_PASSPHRASE_HERE</gpg.passphrase>
</properties>
</profile>
</profiles>
<servers>
<server>
<id>repository.apache.org</id>
<username>${RM_USER}</username>
<password>ENCRYPTED_PASSWORD_HERE</password>
</server>
</servers>
</settings> |
| Info | ||
|---|---|---|
| ||
Steps to configure and encrypt Maven passwords are available at: There are other ways to ensure your PGP key is available for signing as well. |
Git Settings
During the RM process, git will use your code signing private key in order to sign commits and tags. To facilitate this, configure the git user.signingkey property, such as:
...
| language | bash |
|---|
...