Setting up the signing keys
Before you do a release you'll need a signing key that is registered with apache. If you already have one, you can skip this section. Otherwise, here are the steps to do:
- use gpg2 --gen-key to generate a new key. Make sure that the key size is 4096 bits, the key doesn't expire, and use CODE SIGNING KEY for the comment.
- Now you need to register your key at http://pgp.mit.edu/ using the output of gpg --armor --export <keyid>
- Add your key to the KEYS file:
- svn co https://dist.apache.org/repos/dist/release/zookeeper/ zookeeper_dist
- cd zookeeper_dist.
- edit the KEYS file. See the top of the KEYS file for instructions on how to edit this file
- svn ci
- Upload digest to your account on id.apache.org (gpg2 --fingerprint)
Useful Links and Background:
Apache infra page: https://cwiki.apache.org//confluence/display/INFRA/Index
Apache self-service page - now allows for forcing git repo sync: https://selfserve.apache.org/
- when updating JIRA be sure to use the "batch change" operator under the "tools" menu. Disable email notifications when using batch change, subsequent to large JIRA operations (moving issues btw releases, closing JIRAs after a release, etc...) send email to the dev@ list detailing the changes.
Make sure your settings.xml in ~/.m2 contain logins for apache repos, and your signing key is published and configured here.
Please follow the instructions here to encrypt your Apache credentials: https://maven.apache.org/guides/mini/guide-encryption.html
For a more detailed steps, check out How To Release page for Hadoop. It is used as a base.
Before the release, run the following smoke tests (at least).
- Run unit tests on different hardwares: mac, windows, linux.
Checkout a fresh new clone
Check out a fresh new copy of the repository:
Test your docker environment
We are going to perform the full procedure using docker, this way we are sure that the binary artifacts are built using a know environment:
- Java 8
Enter the docker environment (this should work on Mac and on Linux).
In some environment with SELinux you have to enable "permissive mode" ("sudo setenforce Permissive")
Build Java anc C sources, run tests and test GPG signature
Manage JiRA issues
- In Jira, move open issues to next version. Disable mail notifications for this bulk change. If you are in doubt as on email@example.com mailing list before proceedig
Create a branch (when you create X.Y.0 and move master to X.Y+1.0)
Create a branch for X.Y.Z (the current release candidate)
That command will:
- change the version in master branch
- create a new release branch
The goal "branch" in Maven Release plugin does not support "preparationGoals" and "completionGoals" so we have to to manually change the version inside the C-client
So now you have to set the version in zookeeper-client/zookeeper-client-c
In 3.7.0 we have:
Then push a commit with the fix.
It is now time to create CI jobs for the new branch, if you do not have permissions to edit CI configuration just send an email to firstname.lastname@example.org
Preparation for minor release, say 3.6.1
If you are releasing a version that is not the first one of a main branch just checkout the release branch
Create a work branch for the release
If you are releasing a version that is not the first one of a main branch just checkout the release branch
Create release notes on release branch
Update the copyright years in NOTICE.txt if it's outdated.
Update zookeeper-docs/src/main/resources/markdown/releasenotes.md with release notes for this release. You can get the HTML by following the "Release Notes" link for the relevant release on thehttps://issues.apache.org/jira/projects/ZOOKEEPER?selectedItem=com.atlassian.jira.jira-projects-plugin%3Arelease-page&status=unreleased tab in Jira. Note that you need to exclude the won't fix or invalid tickets.
Hints for editing releasenotes.md (using vim) - you can skip 'c' parameter, it tells vim to ask for all replacement:
Push these changes to the $WORK_BRANCH (release-3.6.0) and to the $RELEASE_BRANCH (branch-3.6) (no review is needed at this point)
In case you have to run more than one RC you have to update the release notes every time and push them to those two branches.
Build and check that the release branch is in good shape
Run spotBugs, checkstyle, rat and OWASP checks:
Perform a release on branch (for major and minor release) (remove SNAPSHOT and create tag)
The command above perform the following activities:
- removes -SNAPSHOT in all of the pom.xml files (the version is RELEASE_VERSION)
- build the project (even the C client and contrib packages)
- ensures the correct version in C client files (configure.ac, CMakelink.txt, zookeeper_version.h)
- created a signed tag
- bump the branch to the next minor version (RELEASE_DEVELOPMENT_VERSION)
Preparing the release candidate (prepare stage artifacts for VOTE)
Using the same directory in which you run "release:prepare" perform these are steps
Maven Release Plugin uses release.properties files prepared by release:prepare
- It checks out a new copy of the repository at gitsha pointed by $TAG (like release-3.6.0-0) inside zookeeper/target/checkout
There is no need to build the C client or "contrib" packages
- You should have the SIGNED jar files and also the tarballs built. The two tarball is available at:
- This command also staged all of the maven artifacts to repository.apache.org
Self validate the Release candidate
Check that release file looks ok - e.g. install it and run examples from tutorial.
- untar the release artifact in a test directory
- try to build the sources with
- validate the contents of the staging Maven repository
- test the staged binaries with a JDK8 runtime
Push branches and release tag to upstream
Push the following:
- Work branch
- Release branch
- Release tag
Close the Maven Repository
- Enter Apache Nexus and do the following:
Click on Log In in the upper right corner. Log in using your apache user name and password.
In the left navigation pane, select Staging Repositories.
- Identify the release candidate you just pushed, by your user name (in parentheses as part of the "Repository" name) and the "Created On" date. Click on the check box to the left of your Repository name to select it. (If you accidentally click on the Repository name itself, another tab will pop open. If so, just close it.)
Click the Close button above the Repository names. This makes your release candidate available at the Staging level.
If you have previously staged an older release candidate with the same version number, and it is still showing in the Repository list, you must select and Drop the old one now.
Confirm that your new release candidate is visible at https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/ with correct file modification dates.
Stage source and binaries to your own Apache home (old procedure)
Copy release files to a public place and ensure they are readable. Note that home.apache.org only supports SFTP, so this may be easier with a graphical SFTP client like Nautilus, Konqueror, etc.
Stage source and binaries to Apache dev repo
Copy release files to a public place and ensure they are readable.
Call for VOTE
Call for a release vote on dev (note dev@ and not user@, the user list is for discussion of released software only) at zookeeper.apache.org. Here is a sample email (from 3.4.6 release):
Subject: [VOTE] Apache ZooKeeper release 3.8.0 candidate 0
This is a release candidate for 3.8.0.
It is a major release and it introduces a lot of new features, most notably:
- Migration of the logging framework from Apache Log4j1 to LogBack
- Read Key/trust store password from file (and other security related
- Restored support for OSGI
- Reduced the performance impact of Prometheus metrics
- Official support for JDK17 (all tests are passing)
- Updates to all the third party dependencies to get rid of every known CVE.
The full release notes is available at:
*** Please download, test and vote by February 28th 2022, 23:59 UTC+0. ***
Maven staging repo:
The release candidate tag in git to be voted upon: release-3.8.0-1
ZooKeeper's KEYS file containing PGP keys we use to sign the release:
The staging version of the website is:
Should we release this candidate?
Release Manager Signature
Cancelling a Release Candidate
In case you get a -1 you have to work together with the community in order to see if that vote is a real blocker for the release or not.
In case it is a real blocker we have to fix the problem and cherry-pick the fix to $BRANCH_NAME and to $WORK_BRANCH.
Send an explicit email in answer to the VOTE thread and state that current RC VOTE is cancelled.
"Drop" the Maven Staging Repository at repository.apache.org.
Then when you are okay restart from "Perform a release on branch" step.
In order to create another release candidate, you need to revert the latest two commits on the work and the release branch.
This must be done before cherry-picking the required commits from the master branch. It's also possible to hard reset the latest commit (the SNAPSHOT one), but keep the other intact, because you don't want to delete the previous RC tag.
If you messed up something before publishing a release candidate, feel free to reset the branches to the commit before those "Prepared" commits. You can also delete the RC tag locally and remotely if needed.
Once your work branch is in a good shape, you can force push it.
Once three PMC members have voted for a release, it may be published.
- Answer to the VOTE email thread with a final RESULT email like this one:
- Compare and sync the WORK_BRANCH with the RELEASE_BRANCH, check RELEASE NOTES and Maven Version in pom.xml and in the C client:
Update the version in $BRANCH_NAME (like branch-3.6), the Maven Release plugin pushed it to the WORK_BRANCH, you should cherry pick it (actually they are 2 commits, one to remove SNAPSHOT and one to bump the version) but in case of conflict (if the branch diverged) you can do it with this command but you will have to manually update the C client files
Tag the release. Do it from the release branch and push the created tag to the remote repository, drop the $WORK_BRANCH:
- Check in release files to the distribution server
- After checking in the release, you'll receive an automated email from reporter.apache.org with a link requesting additional details about the release. This form must be completed by a PMC member. If you are a PMC members, follow the link and complete the form. If you are not a PMC member, forward the email to email@example.com and ask for assistance from a PMC member.
The release directory usually contains just two releases, the most recent from two branches, with a link named 'stable' to the most recent recommended version.
- Release the Maven artifacts on Apache Nexus:
- Click on 'Staging repositories'.
- Select the repository corresponding to the release candidate.
- Click on the 'Release' button and follow instructions.
- Wait 24 hours for release to propagate to mirrors.
Prepare to edit the website. ZooKeeper uses Jekyll/Markdown with gitpubsub. See WebSiteSetup for general instructions and tool setup/prerequisites.
Copy the new release documentation into the _released_docs directory
- Change directory to src/main/resources/markdown
- Update the release news in releases.md
- Update the nav panel in html/header.html and documentation.md to include X.Y.Z
Stage changes to git
Regenerate the site, review it, then commit it. (install Jekyll/pygments if you haven't already - see the README.md in website branch)
Deploy: you are now ready to deploy the changes to the public website using gitpubsub. Using the same repo as the previous step:
If the apache zookeeper website does not update, check if the commit is mirrored into github.com/apache/zookeeper asf-site branch
Send announcements to the user and developer lists once the site changes are visible. Here is a sample email:
- In Jira, ensure that only issues in the "Fixed" state have a "Fix Version" set to release X.Y.Z.
- In Jira, "release" the version. Visit the "Administer Project" page, then the "Manage versions" page. You need to have the "Admin" role in ZooKeeper's Jira for this step and the next.
- In Jira, close issues resolved in the release. Disable mail notifications for this bulk change.