This document describes how to release Apache Kafka from trunk.
It is a work in progress and should be refined by the Release Manager (RM) as they come across aspects of the release process not yet documented here.
NOTE: For the purpose of illustration, this document assumes that the version being released is 0.10.0.0 and the following development version will become 0.10.1.0.
- Prepare release plan in the wiki, notifying the community the overall plan and goals for the release (For example: Release Plan 0.10.0)
Vote on release plan: Release manager should start a vote in firstname.lastname@example.org, asking the community to approve the release plan. You will need to join kafka-clients google group at https://groups.google.com/forum/#!forum/kafka-clients in order to send out email to email@example.com
Go over JIRA for the release and make sure that blockers are marked as blockers and non-blockers are non-blockers. This JIRA filter may be handy:
It is important that between the time that the release plan is voted to the time when the release branch is created, no experimental or potentially destabilizing work is checked into the trunk. While it is acceptable to introduce major changes, they must be thoroughly reviewed and have good test coverage to ensure that the release branch does not start of being unstable. If necessary the RM can discuss if certain issues should be fixed on the trunk in this time, and if so what is the gating criteria for accepting them.
- RM must have gpg keys with the public key publicly available to validate the authenticity of the Apache Kafka release: If you haven't set up gpg key, set up one using 4096 bit RSA (http://www.apache.org/dev/release-signing.html). Make sure that your public key is uploaded to one of the public servers (http://www.apache.org/dev/release-signing.html#keyserver). Also, add your public key to https://github.com/apache/kafka-site/blob/asf-site/KEYS
- Make sure the documentation (docs/documentation.html, docs/quickstart.html,docs/api.html) are all referring to the next release and links. If this is a major or minor release #, it's a good idea to make this change now. If you end up doing it after cutting branches, be sure the commit lands on both trunk and your release branch. Note that this must be done before generating any artifacts because these docs are part of the content that gets voted on.
- Make sure you are working with a clean repo (i.e. identical to upstream - no changes in progress). If needed clone a fresh copy
- git checkout trunk
- Check that current HEAD points to commit on which you want to base new release branch. Checkout particular commit if not.
- git branch 0.10.0
- git push apache 0.10.0
- Check that the branch was correctly propagated to Apache using the webui: https://gitbox.apache.org/repos/asf?p=kafka.git
- Modify the version in trunk to bump to the next one "0.10.1.0-SNAPSHOT" in gradle.properties, tests/kafkatest/__init__.py, tests/kafkatest/version.py, streams/quickstart/pom.xml, streams/quickstart/java/pom.xml, and streams/quickstart/java/src/main/resources/archetype-resources/pom.xml. Commit and push to trunk in apache. This is also needed in the non-trunk branch for a bug-fix release.
- Modify the DEFAULT_FIX_VERSION to 0.10.1.0 in kafka-merge-pr.py in the trunk branch. This is not needed in the non-trunk branch for bug-fix release.
- Create new Jenkins Build job that validates the new branch (Java 8 is enough)
Send email announcing the new branch:
Create Release Artifacts
- Run the `release.py` script in the root of the kafka repository and follow the instructions. NOTE that if you are releasing a version prior to 1.0.x, you need to have minor edits on the script to change the three-digits pattern checking to four-digits parttern.
Website update process
We should improve the release script to include these steps. In the meantime, for new releases:
- Make sure the documentation (docs/documentation.html, docs/quickstart.html,docs/api.html) are all referring to the correct release and links.
- git clone https://git-wip-us.apache.org/repos/asf/kafka-site.git
- git checkout asf-site
- The releaseTarGzAll target should auto-generate the configuration docs for broker/producer/consumer in ./core/build/distributions/kafka_2.11-0.10.0.0-site-docs.tgz. Untar the file and rename site-docs to 0100 (or if the latter already exists, replace its contents).
- Copy release javadoc to 0100/ (for bug-fix releases, only do this after the vote has passed to avoid showing an unreleased version number in the published javadocs)
- If this is a major/minor release, update the list of legacy paths in
If need to roll a new RC
- Go to https://repository.apache.org/#stagingRepositories, find the uploaded artifacts and drop it.
- Go back to the beginning - don't forget to bump the RC number.
After the vote passes
- Remember: at least 3 days, 3 +1 from PMC members (committers are not enough!) and no -1.
Send a vote closing email:
- Create a new tag for the release, on the same commit as the voted rc tag and push it:
- Use "git show 0.10.0.0-rc6" to find the commit hash of the tag
- git tag -a 0.10.0.0 <commit hash>
- git push apache 0.10.0.0
- Merge the last version change / rc tag into the release branch and bump the version to 0.10.0.1-SNAPSHOT
- git checkout 0.10.0
- git merge 0.10.0.0-rc6
- Update version on the branch to 0.10.0.1-SNAPSHOT
- git commit -a
- git push apache 0.10.0
- Mark the version as released in Kafka JIRA (from JIRA administration panel, select versions and scroll mouse towards the end of the line for the particular version. From the dropdown list, select release and set the date).
- Upload all artifacts, release notes, and docs to https://dist.apache.org/repos/dist/release/kafka (a SVN repo, using Apache committer id/passwd):
- Only PMC members can upload to the `release` directory, so if the RM is not in the PMC, they can upload the files to https://dist.apache.org/repos/dist/dev/kafka instead and ask a PMC member to move them to the release directory
- svn co https://dist.apache.org/repos/dist/release/kafka kafka-release
- create a new directory for the release (for example kafka-release/0.10.0.0)
- copy the release artifacts from the latest RC (the ones who were in your people.apache.org directory) to the new release directory
- Add the directory to SVN: svn add 0.10.0.0
# Optionally change KEYS file in case that you've added your key for the first time
svn commit -m "Release 0.10.0.0"
- Go to https://repository.apache.org/#stagingRepositories, find the uploaded artifacts and release this (this will push to maven central)
- Wait for about a day for the artifacts to show up in apache mirror and maven central.
- Update the website:
- git clone https://gitbox.apache.org/repos/asf/kafka-site.git
- git checkout asf-site
- If it's a feature release:
- Update documentation.html to include the new documentation link (e.g. 0100/documentation.html)
- Update protocol.html to include the new protocol guide link (e.g. 0100/protocol.html).
- Update quickstart.html to include the new quickstart link
- Update introduction.html to include the new introduction link
- Update downloads.html to include the new download links from mirrors and change last release to use archive. Also add a paragraph with a brief feature introduction for the release.
- git commit -am ".."
- git push origin asf-site
- Send out an announcement email. You will need to use your apache email address to send out the email (otherwise, it won't be delivered to firstname.lastname@example.org).
- Log into people.apache.org with your apache id.
- Include a paragraph in the announcement email like: "According to git shortlog <number_of_contributors> people contributed to this release: <contributors>" where:
- number_of_contributors is determined via `git shortlog -sn --no-merges <previous_release_tag>..<current_release_tag> | wc -l` (eg `git shortlog -sn --no-merges 0.8.2.2..0.9.0.0 | wc -l`)
- contributors is determined via: `git shortlog -sn --no-merges <previous_release_tag>..<current_release_tag> | cut -f2 | tr '\n' ',' | sed -e 's/,/, /g'` (eg `git shortlog -sn --no-merges 0.8.2.2..0.9.0.0 | cut -f2 | sort --ignore-case | tr '\n' ',' | sed -e 's/,/, /g'`)
- cat mail.txt|mail -s "[ANNOUNCE] ..." email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org
note: you need to be subscribed to `email@example.com` with you apache email address – otherwise it bounces back
Template (maybe update Scala versions accordingly):
- The PMC member who committed the release artifacts to the SVN dist repository should add the release data to https://reporter.apache.org/addrelease.html?kafka (they will get a notification asking them to do this after the svn commit).
- PMC member should double check if older releases should be archived (cf. http://www.apache.org/legal/release-policy.html#when-to-archive). This includes updating `download.html`