Summary

Apache NiFi releases involve a Release Manager and members of the project community in various roles. The process involves both technical and procedural steps.

The Release Manager is a member of the Project Management Committee and oversees the process of preparing, reviewing, and publishing versions in coordination with community members.

Release Process Steps

Community member suggests a release timeline through a discussion email thread on dev@nifi.apache.org
A member of the PMC volunteers to act as the Release Manager for the version
The Release Manager validates the source branch and stages the distribution artifacts for a Release Candidate build
The Release Manager sends a vote email thread for the Release Candidate build to dev@nifi.apache.org
Community members verify and vote to approve or reject the Release Candidate build
1. The Release Manager can cancel a vote thread for a Release Candidate build
2. The Release Manager identifies issues for resolution and prepares a new Release Candidate build
The Release Manager publishes the distribution artifacts for an approved Release Candidate build

Prerequisites

Git
GnuPG with published singing key
Java Development Kit 21
Maven settings configured with server entries for Apache Nexus Repository

Maven Settings

<settings>
  <servers>
    <server>
      <id>apache.releases.https</id>
      <username>APACHE_USERNAME</username>
      <password>ENCRYPTED_APACHE_PASSWORD</password>
    </server>
    <server>
      <id>repository.apache.org</id>
      <username>APACHE_USERNAME</username>
      <password>ENCRYPTED_APACHE_PASSWORD</password>
    </server> 
  </servers>
</settings>

Release Candidate Build

The Release Manager is responsible for creating, signing, and staging artifacts for a Release Candidate build.

Version Tracking

Create an Apache NiFi Jira issue for tracking the release process with a Fix Version field set to the Release Version
Create a new version under Apache NiFi Jira Releases with the Version name set to the next minor version
Create a new version section on the project Release Notes page highlighting notable features and fixes

Build Variables

The build process includes several variables that are specific to each Release Candidate build.

variables.sh

#!/bin/bash

# Apache NiFi Jira issue for tracking the release process
JIRA_ISSUE=NIFI-00000

# Release Candidate number for each build
CANDIDATE_NUMBER=1

# Release version number
RELEASE_VERSION=2.0.0

# Next version number applied after the release process
NEXT_SNAPSHOT_VERSION=2.1.0-SNAPSHOT

# Apache Git username for committing changes
export GIT_COMMITTER_NAME=${USER}
export GIT_USER=${USER}

# Apache Git email address for committing changes
export GIT_COMMITTER_EMAIL="${USER}@apache.org"

# GPG key identifier for signing artifacts and commits
GIT_SIGNING_KEY=$(gpg --list-secret-keys --keyid-format LONG ${GIT_COMMITTER_EMAIL}|grep SC|cut -d \/ -f 2|cut -d ' ' -f 1)

# Working directory for building and staging distribution artifacts
WORK_DIR=~/releases/nifi-work
ARTIFACTS_DIR=${WORK_DIR}/artifacts
DIST_DIR=${WORK_DIR}/dist

Build Configuration

The build configuration consists of properties that apply to various steps in the release process.

configuration.sh

#!/bin/bash

# Load build variables
source ./variables.sh

# Set artifact identifier for build elements
ARTIFACT_ID=nifi

# Set Git properties
GIT_AUTHOR="${GIT_COMMITTER_NAME} <${GIT_COMMITTER_EMAIL}>"
GIT_REPO=nifi

# Set Git source location and branch for building release candidate artifacts
GIT_ORIGIN_URL=https://github.com/${GIT_USER}/${GIT_REPO}.git
GIT_SOURCE_BRANCH=main

# Set Git upstream location for committing changes
GIT_UPSTREAM_URL=https://gitbox.apache.org/repos/asf/${GIT_REPO}.git

# Set Git tag and branch names
GIT_CANDIDATE_TAG="${GIT_REPO}-${RELEASE_VERSION}-RC${CANDIDATE_NUMBER}"
GIT_CANDIDATE_BRANCH="${JIRA_ISSUE}-RC${CANDIDATE_NUMBER}"
GIT_RELEASE_TAG="rel/nifi-${RELEASE_VERSION}"

# Set Subversion distribution locations
DIST_DEV_REPO=https://dist.apache.org/repos/dist/dev/nifi/
DIST_RELEASE_REPO=https://dist.apache.org/repos/dist/release/nifi/

Checkout Sources

Cloning the source repository and checking out the source branch provides the baseline for the release candidate build.

checkout-sources.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Create work directory
mkdir -p ${WORK_DIR}
cd ${WORK_DIR}

# Remove previous clones when necessary
rm -rf ${GIT_REPO}

# Clone source repository
git clone ${GIT_ORIGIN_URL}

# Checkout source branch as candidate branch
cd ${GIT_REPO}
git checkout -b ${GIT_CANDIDATE_BRANCH} ${GIT_SOURCE_BRANCH}

Version Sources

Set release version and commit changes to local release candidate branch.

version-sources.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Change to working cloned directory
cd ${WORK_DIR}/${GIT_REPO}

# Set Release Version
./mvnw versions:set -DgenerateBackupPoms=false -DnewVersion=${RELEASE_VERSION}

# Commit Release Version changes
git commit --gpg-sign=${GIT_SIGNING_KEY} -a --author "${GIT_AUTHOR}" -m "${JIRA_ISSUE} Updated version to ${RELEASE_VERSION}"

Stage Artifacts

Build and sign artifacts then deploy to the Nexus Repository. This step can take over an hour based on the number of artifacts.

stage-artifacts.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Change to working cloned directory
cd ${WORK_DIR}/${GIT_REPO}

# Build and deploy artifacts to Nexus Repository with required release profile
./mvnw deploy -P apache-release -DskipTests -Dgpg.keyname=${GIT_COMMITTER_EMAIL}

Stage Binaries

Stage binary archives for signing.

stage-binaries.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Create Artifacts directory
mkdir -p ${ARTIFACTS_DIR} 

# Define Source Release Binary and signatures
SOURCE_RELEASE_ZIP=${ARTIFACT_ID}-${RELEASE_VERSION}-source-release.zip
SOURCE_RELEASE_ZIP_ASC=${SOURCE_RELEASE_ZIP}.asc
SOURCE_RELEASE_ZIP_SHA512=${SOURCE_RELEASE_ZIP}.sha512
SOURCE_DIR=${WORK_DIR}/${GIT_REPO}

# Copy Source Release files to artifacts directory
cp ${SOURCE_DIR}/target/${SOURCE_RELEASE_ZIP} ${ARTIFACTS_DIR}
cp ${SOURCE_DIR}/target/${SOURCE_RELEASE_ZIP_ASC} ${ARTIFACTS_DIR}
cp ${SOURCE_DIR}/target/${SOURCE_RELEASE_ZIP_SHA512} ${ARTIFACTS_DIR}

# Copy binary archives to artifacts directory
cp ${SOURCE_DIR}/minifi/minifi-assembly/target/minifi-${RELEASE_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp ${SOURCE_DIR}/minifi/minifi-toolkit/minifi-toolkit-assembly/target/minifi-toolkit-${RELEASE_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp ${SOURCE_DIR}/nifi-assembly/target/nifi-${RELEASE_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp ${SOURCE_DIR}/nifi-registry/nifi-registry-assembly/target/nifi-registry-${RELEASE_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp ${SOURCE_DIR}/nifi-registry/nifi-registry-toolkit/nifi-registry-toolkit-assembly/target/nifi-registry-toolkit-${RELEASE_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp ${SOURCE_DIR}/nifi-stateless/nifi-stateless-assembly/target/nifi-stateless-${RELEASE_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp ${SOURCE_DIR}/nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-${RELEASE_VERSION}-bin.zip ${ARTIFACTS_DIR}

Sign Binaries

Sign binary archives and generate hashes for subsequent publication.

sign-binaries.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Change to artifacts directory
cd ${ARTIFACTS_DIR}

# Sign binary archives
gpg -a -b --local-user ${GIT_SIGNING_KEY} --yes --digest-algo=SHA512 minifi-${RELEASE_VERSION}-bin.zip
gpg -a -b --local-user ${GIT_SIGNING_KEY} --yes --digest-algo=SHA512 minifi-toolkit-${RELEASE_VERSION}-bin.zip
gpg -a -b --local-user ${GIT_SIGNING_KEY} --yes --digest-algo=SHA512 nifi-${RELEASE_VERSION}-bin.zip
gpg -a -b --local-user ${GIT_SIGNING_KEY} --yes --digest-algo=SHA512 nifi-registry-${RELEASE_VERSION}-bin.zip
gpg -a -b --local-user ${GIT_SIGNING_KEY} --yes --digest-algo=SHA512 nifi-registry-toolkit-${RELEASE_VERSION}-bin.zip
gpg -a -b --local-user ${GIT_SIGNING_KEY} --yes --digest-algo=SHA512 nifi-stateless-${RELEASE_VERSION}-bin.zip
gpg -a -b --local-user ${GIT_SIGNING_KEY} --yes --digest-algo=SHA512 nifi-toolkit-${RELEASE_VERSION}-bin.zip

# Generate SHA-512 hash for binary archives
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha512' -- minifi-${RELEASE_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha512' -- minifi-toolkit-${RELEASE_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha512' -- nifi-${RELEASE_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha512' -- nifi-registry-${RELEASE_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha512' -- nifi-registry-toolkit-${RELEASE_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha512' -- nifi-stateless-${RELEASE_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha512' -- nifi-toolkit-${RELEASE_VERSION}-bin.zip

Distribute Binaries

Distribute binary artifacts to to the development distribution location using Subversion.

distribute-binaries.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Change to distribution directory
mkdir -p ${DIST_DIR}
cd ${DIST_DIR}

# Checkout development distribution repository to local directory
svn checkout ${DIST_DEV_REPO} ${ARTIFACT_ID}
RELEASE_VERSION_DIR=${ARTIFACT_ID}-${RELEASE_VERSION}
RELEASE_DIR=${ARTIFACT_ID}/${RELEASE_VERSION_DIR}

# Copy binary artifacts to version directory
cd ${ARTIFACT_ID}
cp -r ${ARTIFACTS_DIR} ${RELEASE_VERSION_DIR}

# Commit and push binary artifacts to Subversion
svn update
svn add ${RELEASE_VERSION_DIR}
svn commit -m "${JIRA_ISSUE} Staging artifacts for ${RELEASE_VERSION_DIR}-RC${CANDIDATE_NUMBER}" ${RELEASE_VERSION_DIR}

Push Sources

Create source tag and push both tag and branch to upstream source repository.

push-sources.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Change to working cloned directory
cd ${WORK_DIR}/${GIT_REPO}

# Tag and sign version
git tag --sign --local-user=${GIT_SIGNING_KEY} -m "${JIRA_ISSUE} Tagged Release Candidate ${CANDIDATE_NUMBER}" ${GIT_CANDIDATE_TAG}

# Add remote repository location
git remote add upstream ${GIT_UPSTREAM_URL}

# Push candidate branch and candidate tag
git push upstream ${GIT_CANDIDATE_BRANCH}
git push upstream ${GIT_CANDIDATE_TAG}

Prepare Email

Prepare email for voting on release candidate build. The REPOSITORY_NUMBER must match the value from the Apache Nexus Repository.

prepare-email.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Get Apache Nexus Repository number from staging artifacts
REPOSITORIES_URL=https://repository.apache.org/content/repositories/
REPOSITORY_NAME=$(curl -s ${REPOSITORIES_URL}|grep orgapachenifi|cut -d '>' -f 3|cut -d / -f 1)

# Get Jira Version and Issues
VERSIONS_JSON=$(curl -s https://issues.apache.org/jira/rest/api/2/project/NIFI/versions)
JIRA_VERSION=$(echo ${VERSIONS_JSON}|jq -r ".[] | select(.name==\"${RELEASE_VERSION}\") | .id" )
PROJECT_ID=$(echo ${VERSIONS_JSON}|jq -r ".[] | select(.name==\"${RELEASE_VERSION}\") | .projectId" )
JIRA_RELEASE_NOTES="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=${PROJECT_ID}&version=${JIRA_VERSION}"

# Get Total Issues
ISSUES_JSON=$(curl -s https://issues.apache.org/jira/rest/api/2/version/${JIRA_VERSION}/relatedIssueCounts)
TOTAL_ISSUES=$(echo ${ISSUES_JSON}|jq -r ".issuesFixedCount")

# Set total number of Jira issues changed
TOTAL_ISSUES=0

# Set SHA-512 hash of source release archive
SHA512_HASH=`cat ${WORK_DIR}/${GIT_REPO}/target/${ARTIFACT_ID}-${RELEASE_VERSION}-source-release.zip.sha512`

# Change to cloned working directory
cd ${WORK_DIR}/${GIT_REPO}

# Get current Git commit ref
GIT_COMMIT_ID=`git rev-parse HEAD`
PROJECT_LABEL="Apache NiFi"

echo """
[VOTE] Release ${PROJECT_LABEL} ${RELEASE_VERSION} (RC${CANDIDATE_NUMBER})

Team,

I am pleased to be calling this vote for the source release of ${PROJECT_LABEL} ${RELEASE_VERSION}.

Please review the following guide for how to verify a release candidate build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on the Apache Distribution Repository:

${DIST_DEV_REPO}${ARTIFACT_ID}-${RELEASE_VERSION}

The build artifacts are available on the Apache Nexus Repository:

${REPOSITORIES_URL}${REPOSITORY_NUMBER}/

Git Tag: ${GIT_REPO}-${RELEASE_VERSION}-RC${CANDIDATE_NUMBER}
Git Commit ID: ${GIT_COMMIT_ID}
GitHub Commit Link: https://github.com/apache/${GIT_REPO}/commit/${GIT_COMMIT_ID}

Hashes of ${ARTIFACT_ID}-${RELEASE_VERSION}-source-release.zip

SHA512: ${SHA512_HASH}

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/${GIT_COMMITTER_NAME}.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: ${TOTAL_ISSUES}

${JIRA_RELEASE_NOTES}

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version${RELEASE_VERSION}

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build from source, and test. Then please vote:

[] +1 Release this package as ${ARTIFACT_ID}-${RELEASE_VERSION}
[] +0 no opinion
[] -1 Do not release this package because...
"""

Release Candidate Voting

Project Management Committee members can cast binding votes that decide whether the release is approved.

Committers and community members can cast non-binding votes that inform the Release Manager and PMC members about the release candidate status.

Release Candidate Cancellation

The Release Manager can cancel a vote in response to negative findings. Cancelling a vote requires sending an email with the word CANCEL in the subject line.

[CANCEL][VOTE] Release Apache NiFi 2.0.0-RC1

Release Candidate Approval

Approving a Release Candidate build requires at least 3 binding positive votes from project PMC members and more positive votes than negative votes.

The Release Manager sends an email to the project Developers Mailing List with the vote results. The Vote Thread ID can be accessed from the Developer Mailing List archives.

[RESULT][VOTE] Release Apache NiFi 2.0.0-RC1

Apache NiFi Community,

I am pleased to announce that the 2.0.0 release of Apache NiFi passes:

  X +1 (binding) votes
  Y +1 (non-binding) votes
  0 0 votes
  0 -1 votes

Thanks to all who helped make this release possible!

Here is the vote thread: https://lists.apache.org/thread/${VOTE_THREAD_ID}

Release Candidate Repository Updates

After a positive release candidate vote, the source branch should be updated.

Create Signed Tag

Create a signed Git tag and push to the source repository.

create-tag.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Change to source directory
cd ${WORK_DIR}/${GIT_REPO}

# Create signed Git tag
git tag -u ${GIT_SIGNING_KEY} -s ${GIT_RELEASE_TAG} -m "${JIRA_ISSUE} Released version ${RELEASE_VERSION}" ${GIT_CANDIDATE_TAG}

# Push tag to source repository
git push upstream ${GIT_RELEASE_TAG}

Merge Changes

Merge version changes from release branch to main branch.

merge-changes.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Change to source directory
cd ${WORK_DIR}/${GIT_REPO}

# Fetch current upstream branch
git fetch upstream

# Rebase upstream changes
git checkout ${GIT_SOURCE_BRANCH}
git rebase upstream/${GIT_SOURCE_BRANCH}

# Merge changes
git merge --no-ff ${GIT_CANDIDATE_BRANCH} --gpg-sign=${GIT_SIGNING_KEY} -m "${JIRA_ISSUE} Merged ${GIT_CANDIDATE_BRANCH}"

# Push changes to source repository
git push upstream ${GIT_SOURCE_BRANCH}

Set Next Version

Set next snapshot version and push to source repository.

set-next-version.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Change to source directory
cd ${WORK_DIR}/${GIT_REPO}

# Set Next Snapshot Version
./mvnw versions:set -DgenerateBackupPoms=false -DnewVersion=${NEXT_SNAPSHOT_VERSION}

# Commit Version
git commit --gpg-sign=${GIT_SIGNING_KEY} -a --author "${GIT_AUTHOR}" -m "${JIRA_ISSUE} Updated version to ${NEXT_SNAPSHOT_VERSION}"

# Push Updated Version
git push upstream

Release Binary Artifacts

The binary release process consists of publishing artifacts through standard distribution channels.

Distribute Binary Release Artifacts

Move the binary release artifacts from the development directory to the release directory.

distribute-binary-artifacts.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Change to distribution directory
cd ${DIST_DIR}

# Set locations
DIST_DEV_URL=${DIST_DEV_REPO}/${ARTIFACT_ID}-${RELEASE_VERSION}
DIST_RELEASE_URL=${DIST_RELEASE_REPO}/${RELEASE_VERSION}

# Move binary artifacts with Subversion
svn move -m "${JIRA_ISSUE}" ${DIST_DEV_URL} ${DIST_RELEASE_URL}

Publish Binary Artifacts through Nexus

The Apache Nexus Repository containing the binary artifacts should be marked as Released for subsequent distribution through Maven Central.

Update Website Versions

The Apache NiFi website includes download links for the released version and documentation generated based on versioned binaries.

Clone Website Repository

The Apache NiFi website repository contains the source for the project website, which can be updated using a commit to the main branch.

Update Website Version References

The Apache NiFi website repository has a current version configuration property that should be updated to reflect the released version.

The project configuration in config.toml has several version properties that should be updated.

config.toml

currentProjectVersion = "2.1.0"
currentProjectVersionReleased = "2024-12-23"

Verify Website Changes

Project website changes should be verified prior to pushing to the repository.

The project README.md provides the steps for preparing and building the website for local verification.

Push Website Changes

Commit and push the changes to the main branch with a commit message that references the release candidate Jira number.

The GitHub Actions workflow publishes changes for download links and documentation.

Update Release Tracking

Jira releases and release notes should be updated to indicate the date of the release.

Update Release Notes

The Apache NiFi Release Notes page should be updated to indicate the date of the release for the associated version.

Update Jira Releases

The Apache NiFi Releases section in Jira should be updated with the date of the release. The Actions menu associated with the release version has item labeled Release that should be selected to set the released status.

Update Migration Guide

The Apache NiFi Migration Guidance page should be updated with a new section for the associated version when there are notable changes required for moving from a previous version.

Delete Previous Version

Previous release versions should be deleted from the Apache Release Distribution Repository.

delete-previous-version.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Set Previous Version number
PREVIOUS_VERSION="2.0.0"

# Set location
PREVIOUS_RELEASE_URL=${DIST_RELEASE_REPO}/${PREVIOUS_VERSION}

# Delete previous artifacts with Subversion
svn delete -m "${JIRA_ISSUE}" ${PREVIOUS_RELEASE_URL}

Announce Release

Send an email to the Apache Announcements List as well as project mailing lists announcing the new version.

announce@apache.org
users@nifi.apache.org
dev@nifi.apache.org

announce-email.sh

#!/bin/bash

# Load configuration
source ./configuration.sh

# Get Jira Version and Issues
PROJECT_LABEL="Apache NiFi"
VERSIONS_JSON=$(curl -s https://issues.apache.org/jira/rest/api/2/project/NIFI/versions)
JIRA_VERSION=$(echo ${VERSIONS_JSON}|jq -r ".[] | select(.name==\"${RELEASE_VERSION}\") | .id" )
PROJECT_ID=$(echo ${VERSIONS_JSON}|jq -r ".[] | select(.name==\"${RELEASE_VERSION}\") | .projectId" )
JIRA_RELEASE_NOTES="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=${PROJECT_ID}&version=${JIRA_VERSION}"

echo """
[ANNOUNCE] ${PROJECT_LABEL} ${RELEASE_VERSION} Released

The Apache NiFi Team is pleased to announce the release of ${PROJECT_LABEL} ${RELEASE_VERSION}.

Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data.

https://nifi.apache.org

The release artifacts can be loaded from the project website.

https://nifi.apache.org/download/

Maven artifacts have been released and mirrored according to Apache distribution processes.

Issues resolved in ${PROJECT_LABEL} ${RELEASE_VERSION} are listed in Jira Release Notes.
 
${JIRA_RELEASE_NOTES}

Highlights of the release are available on the project wiki.

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes

Thank you,
The Apache NiFi Team
"""

Space shortcuts

Child pages