This page is prepared for Apache Avro committers. You need committer rights to create a new Apache Avro release.
Table of Contents |
---|
Branching
Skip this section if this is NOT the first release in a series (i.e. release X.Y.0).
- Update
CHANGES.txt
to include the release version and date (useUnreleased
for the date if it is unknown) and removeTrunk (unreleased changes)
. Commit these changes to trunk.
No Format svn commit -m "Preparing for release X.Y.Z"
Create a branch for the release series:
No Format svn copy https://svn.apache.org/repos/asf/avro/trunk \ https://svn.apache.org/repos/asf/avro/branches/branch-X.Y -m "Branching for X.Y releases"
- Update
CHANGES.txt
to add back inTrunk (unreleased changes)
. - Update the default version in
share/VERSION.txt
on trunk to X.Y+1.0-SNAPSHOT. Be sure not to leave a trailing newline. Update the version in the Maven POM files to match with
No Format mvn versions:set -DnewVersion=X.Y+1.0-SNAPSHOT -DgenerateBackupPoms=false
Commit these changes to trunk.
No Format svn commit -m "Preparing for X.Y+1.0 development"
Updating Release Branch
These operations take place in the release branch.
Build environment
I often use Docker to build a clean and consistent build environment:
No Format |
---|
# Build a clean docker image from scratch.
DOCKER_RUN_ENTRYPOINT=true DOCKER_BUILD_XTRA_ARGS=--no-cache DOCKER_IMAGE_NAME=avro-build:X.Y-rcN ./build.sh docker
# Run the docker container, automatically mapping ~/.gnupg, ~/.m2 and the current directory to the container.
DOCKER_IMAGE_NAME=avro-build:X.Y-rcN ./build.sh docker
# Omitting DOCKER_IMAGE_NAME will use avro-build-$USER:latest
|
Branching
Skip this section if this is NOT the first release in a series (i.e. release X.Y.0).
Create a release branch for the release series (from the master):
No Format git checkout -b branch-X.Y
Update the version:
No Format mvn versions:set -DnewVersion=X.Y.0 -DgenerateBackupPoms=false
Check out the branch with:
No Format svn co https://svn.apache.org/repos/asf/avro/branches/branch-X.Y
- Update
CHANGES.txt
to include the release version and date (this change must be committed to trunk and any intermediate branches between trunk and the branch being released).- Update the version number in
share/VERSION.txt
to be ''
- X.Y.
- 0'
- . Be sure
- NOT to leave a trailing newline.
- Update the version
mvn versions:set -DnewVersion=No Format - number in
lang/js/package.json
to X.Y.
- Update the version number in
Commit these changes.
No Format svn commit -m "Preparing for release X.Y.Z"
- Add the fix version X.Y.Z to the Avro JIRA
If not already done, merge desired patches from trunk into the branch and commit these changes. You can find the revision numbers using
svn log CHANGES.txt
in the branch and in trunk.No Format cd branch-X.Y svn merge -rR1:R2 ../trunk . svn commit -m "Merge -r R1:R2 from trunk to X.Y branch. Fixes: AVRO-A, AVRO-B."
- For each patch merged, change the fix version for the JIRA issue to be X.Y.Z
- Go through CHANGES.txt, JIRA, and svn log to be sure that the issues included in the branch match in each location, then update the date in CHANGES.txt to be today.
- Update the version number in
share/VERSION.txt
to be ''X.Y.Z''. Be sure not to leave a trailing newline. Update the version in the Maven POM files to match with
No Format mvn versions:set -DnewVersion=X.Y.Z -DgenerateBackupPoms=false
Update the version number in
lang/c/version.sh
(the variableslibavro_micro_version
,libavro_interface_age
andlibavro_binary_age
) according to the libtool versioning rules as described in that file. Note the libtool version number is completely unrelated to the Avro release version number.Commit these changes.
No Format svn commit -m "Preparing to build X.Y.Z"
Tag the release candidate (R is the release candidate number):
No Format svn copy https://svn.apache.org/repos/asf/avro/branches/branch-X.Y \ https://svn.apache.org/repos/asf/avro/tags/release-X.Y.Z-rcR -m "Avro X.Y.Z-rcR release."
Building
Unless you have set up the required dependencies to build Avro for all languages, the following should be run inside a Docker container (see BUILD.txt for instructions).
Build the release & run unit tests.
No Format ./build.sh clean dist test
- Check that release files look ok - e.g. unpack the sources and run tests.
Sign the release (see Step-By-Step Guide to Mirroring Releases for more information).
No Format ./build.sh sign
To sign a release, your key must be present in the dist/KEYS file. See the Apache guide to Signing Releases for more details. Also, if you've updated the dist/KEYS file, be sure to update the public Apache KEYS file as well:
Code Block |
---|
ssh people.apache.org
cd /www/www.apache.org/dist/avro
svn up
|
Copy release files to the public staging area https://dist.apache.org/repos/dist/dev/avro/
No Format svn co https://dist.apache.org/repos/dist/dev/avro/ avro-dev-dist mkdir avro-dev-dist/avro-X.Y.Z-rcR cp -pr avro/dist/* avro-dev-dist/avro-X.Y.Z-rcR cd avro-dev-dist svn add avro-X.Y.Z-rcR svn commit -m "Artifacts for Avro X.Y.Z RCR"
Stage two versions of Java artifacts to the Maven repository:
No Format mvn clean -P dist,sign deploy -DskipTests=true -Dhadoop.version=1 -Dgpg.passphrase=XXX -Davro.version=X.Y.Z mvn clean -P dist,sign deploy -DskipTests=true -Dgpg.passphrase=XXX -Davro.version=X.Y.Z
Find the Staging Repository, and close it.
- Call a release vote on dev at avro.apache.org.
Include the URL of the staging repository.
Publishing
Once three PMC members have voted for a release, it may be published.
- 0
- Update the version number in
pom-XXXX.xml
used for unit tests in thelang/java/maven-plugin/src/test/resources
directory too.
Update the release branch.
No Format git commit -am "Preparing for release X.Y.0" git push apache branch-X.Y
Return to the master branch.
No Format git checkout master
Update the master version to match the next major version:
No Format mvn versions:set -DnewVersion=X.(Y+1).0-SNAPSHOT -DgenerateBackupPoms=false
- Update the version number in
share/VERSION.txt
to be ''X.(Y+1).0-SNAPSHOT'. Be sure NOT to leave a trailing newline. - Update the version number in
lang/js/package.json
to X.(Y+1).0-SNAPSHOT - Update the version number in
pom-XXXX.xml
used for unit tests in thelang/java/maven-plugin/src/test/resources
directory too.
- Update the version number in
Update the master branch
No Format git commit -am "Preparing for X.(Y+1).0 development" git push apache master
Updating Release Branch
These operations take place in the release branch.
Check out the branch with:
No Format git checkout branch-X.Y
Update the version:
No Format mvn versions:set -DnewVersion=X.Y.Z -DgenerateBackupPoms=false
- Update the version number in
share/VERSION.txt
to beX.Y.Z
. Be sure NOT to leave a trailing newline. - Update the version number in
lang/js/package.json
toX.Y.Z
. Update or regeneratelang/js/package-lock.json
. - Update the version number in
pom-XXXX.xml
used for unit tests in thelang/java/maven-plugin/src/test/resources
directory too. - Update the versions in
doc/examples/java-example
anddoc/examples/
.mr
-example
- Update the version number in
Commit the update to the release branch.
No Format git commit -am "Preparing for release X.Y.Z"
- Add the fix version X.Y.Z to the Avro JIRA
If not already done, cherry-pick desired patches from master into the branch and commit these changes.
No Format git checkout branch-X.Y git cherry-pick <commit>
- For each patch merged, change the fix version for the JIRA issue to be X.Y.Z
- Go through JIRA, and git log to be sure that the issues included in the branch match in each location.
Update the version number in
lang/c/version.sh
(the variableslibavro_micro_version
,libavro_interface_age
andlibavro_binary_age
) according to the libtool versioning rules as described in that file. Note the libtool version number is completely unrelated to the Avro release version number.Commit these changes.
No Format git commit -am "Preparing to build X.Y.Z"
Tag the release candidate (R is the release candidate number):
No Format git tag -s release-X.Y.Z-rcR -m "Avro X.Y.Z rcR release."
Push the release tag and branch changes:
No Format git push apache release-X.Y.Z-rcR git push apache branch-X.Y
Building
Unless you have set up the required dependencies to build Avro for all languages, the following should be run inside a Docker container (see BUILD.md
for instructions).
Build the release & run unit tests. It is strongly recommended to do this inside the docker container.
No Format # To launch a bash shell inside the docker container. ./build.sh docker # Once inside the container. ./build.sh clean test dist
- Check that release files look ok - e.g. unpack the sources and run tests.
Sign the release (see Step-By-Step Guide to Mirroring Releases for more information).
No Format ./build.sh sign # Or if using a non-default key (AVRO-3858): GPG_LOCAL_USER=B0CCBDD6 ./build.sh sign
To sign a release, your key must be present in the dist/KEYS file. See the Apache guide to Signing Releases for more details. Also, if you've updated the dist/KEYS file, be sure to update the public Apache KEYS file as well:
Code Block svn co --depth=files
Tag the release:
No Format svn move https://svn.apache.org/repos/asf/avro/tags/release-X.Y.Z-rcR \ https://svn.apache.org/repos/asf/avro/tags/release-X.Y.Z -m "Avro X.Y.Z release."
Copy release files to the release repository.
No Format svn copy https://dist.apache.org/repos/dist/dev/avro/avro-X.Y.Z-rcR \ https://dist.apache.org/repos/dist/release/avro/avro-X.Y.Z -m "Avro X.Y.Z release."
The release directory usually contains just two releases, the most recent from two branches, with a link named 'stable' to the most recent recommended version.
No Format svn co https://dist.apache.org/repos/dist/release/avro/ avro-release-distarchive-avro/ cd avro-release-dist svn rm avro-A.B.C; rm stable ln -s avro-X.Y.Z stable svn commit -m "Avro X.Y.Z release."
- Publish Java artifacts to the Maven repository:
Find the Staging Repository and release it. Publish Python artifacts to PyPI. To do this you'll need an account on PyPi, and write access to the Avro package - ask the existing owners for permission if you don't have it.
No Format mkdir -p tmp/py cd tmp/py tar xzf ../../dist/py/avro-X.Y.X.tar.gz cd avro-X.Y.Z python setup.py sdist upload
Publish Python3 artifacts to PyPI.
No Format mkdir -p tmp/py3 cd tmp/py3 tar xvf ../../dist/py3/avro-python3-X.Y.Z.tar.gz cd avro-python3-X-Y-Z python3 ./setup.py sdist upload
Publish Ruby artifacts to RubyGems. Again, you'll need an account and you need to be an owner.
No Format gem push dist/ruby/avro-X.Y.Z.gem
Publish JavaScript artifacts to NPM. Again, you'll need an account and you need to be an owner.
No Format npm publish dist/js/avro-js-X.Y.Z.tgz
- Wait 24 hours for release to propagate to mirrors.
Prepare to edit the website.
No Format svn co https://svn.apache.org/repos/asf/avro/site
- Update the documentation links in
author/content/xdocs/site.xml
. - Update the release news in
author/content/xdocs/releases.xml
. Regenerate the site, review it, then commit it. Note that Forrest 0.9 is easiest (not currently in the Docker image), since it doesn't require Java 5.
No Format cd site ant firefox publish/index.html svn commit -m "Updated site for release X.Y.Z."
Copy the new release docs to website and update the
docs/current
link:No Format tar xzf dist/avro-doc-X.Y.Z.tar.gz mv avro-doc-X.Y.Z ../site/publish/docs/X.Y.Z cd ../site/publish svn add X.Y.Z rm current ln -s X.Y.Z current svn commit -m "Adding documentation for release X.Y.Z."
- Send announcements to the user and developer lists once the site changes are visible.
- Update the version number in
share/VERSION.txt
to be ''avro-X.Y.N-SNAPSHOT'', where ''N'' is one greater than the release just made. Update the version in the Maven POM files to match with
No Format mvn versions:set -DnewVersion=X.Y.N-SNAPSHOT -DgenerateBackupPoms=false
- In Jira, ensure that only issues in the "Fixed" state have a "Fix Version" set to release X.Y.Z.
- In Jira, "release" the version. Visit the "Administer Project" page, then the "Manage versions" page. You need to have the "Admin" role in Avro's Jira for this step and the next.
- In Jira, close issues resolved in the release. Disable mail notifications for this bulk change.
See Also
archive-avro/ head -n 14 KEYS This file contains the PGP keys of various developers. Users: pgp < KEYS or gpg --import KEYS Developers: pgp -kxa <your name> and append it to this file. or (pgpk -ll <your name> && pgpk -xa <your name>) >> this file. or (gpg --list-sigs <your name> && gpg --armor --export <your name>) >> this file.
As you can see, the header of the KEYS file has instructions to add your key. Then commit using: svn commit -m "Add key for ..."
Copy release files to the public staging area https://dist.apache.org/repos/dist/dev/avro/
No Format svn co https://dist.apache.org/repos/dist/dev/avro/ avro-dev-dist mkdir avro-dev-dist/avro-X.Y.Z-rcR cp -pr avro/dist/* avro-dev-dist/avro-X.Y.Z-rcR cd avro-dev-dist svn add avro-X.Y.Z-rcR svn commit -m "Artifacts for Avro X.Y.Z RCR"
Stage the default Hadoop 2 version of Java artifacts to the Maven repository:
No Format mvn clean -P dist,sign deploy -DskipTests=true -Davro.version=X.Y.Z
Make sure that you've followed to guide to configure Maven: http://www.apache.org/dev/publishing-maven-artifacts.html
- Find the Staging Repository, and close it.
- Call a release vote on dev at avro.apache.org. Include the URL of the staging repository.
Publishing
Once three PMC members have voted for a release, it may be published. Please prepare in advance your account and request correct permissions to be able to publish in the different services: PyPi, RubyGems, NuGet, npm and CPAN
Tag the release:
No Format git checkout release-X.Y.Z-rcR git tag -s release-X.Y.Z -m "Avro X.Y.Z release." git push apache release-X.Y.Z
Copy release files to the release repository (PMC permissions required):
No Format svn copy https://dist.apache.org/repos/dist/dev/avro/avro-X.Y.Z-rcR \ https://dist.apache.org/repos/dist/release/avro/avro-X.Y.Z -m "Avro X.Y.Z release."
The release directory usually contains just two releases, the most recent from two branches, with a link named 'stable' to the most recent recommended version (PMC permissions required):
No Format svn co https://dist.apache.org/repos/dist/release/avro/ avro-release-dist cd avro-release-dist svn rm avro-A.B.C; rm stable ln -s avro-X.Y.Z stable svn commit -m "Avro X.Y.Z release."
- Publish Java artifacts to the Maven repository:
Find the Staging Repository and release it. Publish Python artifacts to PyPI. To do this you'll need an account on PyPi, and write access to the Avro package - ask the existing owners for permission if you don't have it. Also see the build and twine documentation.
No Format # Install build and twine in the docker image export PATH=~/.local/bin/:$PATH python3 -m pip install --user --upgrade build pip twine mkdir -p tmp/py cd tmp/py tar xvfz ../../dist/py/avro-X.Y.X.tar.gz cd avro-X.Y.Z python3 -m build # build a sdist, and then the wheel from the sdist. twine upload dist/*
Publish Ruby artifacts to RubyGems. Again, you'll need an account and you need to be an owner.
No Format gem push dist/ruby/avro-X.Y.Z.gem
Publish JavaScript artifacts to npm. Again, you'll need an account and you need to be an owner.
No Format npm login npm publish dist/js/avro-js-X.Y.Z.tgz
Publish C# artifacts to NuGet. Again, you'll need an account and you need to be an owner. You also need to generate an additional key to publish.
No Format mkdir -p tmp/csharp cd tmp/csharp tar xvfz ../../dist/csharp/avro-csharp-X.Y.Z.tar.gz dotnet nuget push main/Apache.Avro.X.Y.Z.nupkg -k YOUR_KEY -s https://api.nuget.org/v3/index.json dotnet nuget push codegen/Apache.Avro.Tools.X.Y.Z.nupkg -k YOUR_KEY -s https://api.nuget.org/v3/index.json dotnet nuget push codec/Avro.File.Snappy/Apache.Avro.File.Snappy.X.Y.Z.nupkg -k YOUR_KEY -s https://api.nuget.org/v3/index.json dotnet nuget push codec/Avro.File.BZip2/Apache.Avro.File.BZip2.X.Y.Z.nupkg -k YOUR_KEY -s https://api.nuget.org/v3/index.json dotnet nuget push codec/Avro.File.XZ/Apache.Avro.File.XZ.X.Y.Z.nupkg -k YOUR_KEY -s https://api.nuget.org/v3/index.json dotnet nuget push codec/Avro.File.Zstandard/Apache.Avro.File.Zstandard.X.Y.Z.nupkg -k YOUR_KEY -s https://api.nuget.org/v3/index.json
Publish Perl artifacts to CPAN. Again, you'll need an account and you need to be an owner.
No Format cpan-upload -u YOUR_ID Avro-X.Y.Z.tar.gz
- Wait 24 hours for release to propagate to mirrors.
Prepare to edit the website. THIS SECTION IS OUT OF DATE. TODO!
No Format svn co https://svn.apache.org/repos/asf/avro/site
- Update the documentation links in
author/content/xdocs/site.xml
. - Update the release news in
author/content/xdocs/releases.xml
. Run Apache Forrest using Apache Ant. Example is with forrest installed using homebrew: http://macappstore.org/apache-forrest/
No Format cd site ant -Dforrest.home=/usr/local/Cellar/apache-forrest/0.9/ # Alternatively, using forrest inside the avro-build docker. ant -Dforrest.home=/usr/local/apache-forrest
Copy the new release docs, which are generated at the dist step, to website and update the
docs/current
link:No Format tar xzf dist/avro-doc-X.Y.Z.tar.gz mv avro-doc-X.Y.Z ../site/publish/docs/X.Y.Z cd ../site/publish/docs svn add X.Y.Z rm current ln -s X.Y.Z current svn commit -m "Adding documentation for release X.Y.Z."
- Send an announcement email to the the mailing lists: announce@apache.org, user@avro.apache.org. and dev@avro.apache.org once the site changes are visible. Use your personal `@apache.org` email address or it will be refused.
- In Jira, "release" the version. Visit the Manage versions page. You need to have the "Admin" role in Avro's Jira for this step and the next.
- In Jira, close issues resolved in the release. Disable mail notifications for this bulk change.
- Update the release information in Apache Committee Report Helper (you should receive a reminder by email).
- Create a new release on Github:
- Draft a new release, using the release-X.Y.Z tag created in step 1.
- Release title: Apache Avro X.Y.Z
Post-release steps:
Prepare next release cycle
Update the version where ''N'' is one greater than the release just made:
No Format mvn versions:set -DnewVersion=X.Y.N-SNAPSHOT -DgenerateBackupPoms=false
- Update the version number in
share/VERSION.txt
to be ''X.Y.N-SNAPSHOT'. Be sure NOT to leave a trailing newline. - Update the version number in
lang/js/package.json
to X.Y.N-SNAPSHOT - Update the version number in
pom-XXXX.xml
used for unit tests in thelang/java/maven-plugin/src/test/resources
directory too.
- Update the version number in
Update to the release branch.
No Format |
---|
git commit -am "Preparing for release X.Y.N"
git push apache branch-X.Y |
Update Documentation
- Update the Apache Avro wikipedia page
- Update this guide with the changes or issues you found while releasing the version. Extra points if you can reduce or automate some steps!
Validate published artifacts
You can check the published version of the artifacts is correctly available in the URL or if you want to double check that the artifacts can be downloaded/used you can use the suggested CLI
Java - https://search.maven.org/artifact/org.apache.avro/avro
No Format docker run -it maven:3 mvn dependency:get -Dartifact=org.apache.avro:avro:X.Y.Z
Python 2 / 3 - https://pypi.org/project/avro / https://pypi.org/project/avro-python3
No Format docker run -it python:2 pip install 'avro==X.Y.Z' docker run -it python:3 pip install 'avro==X.Y.Z'
Ruby - https://rubygems.org/gems/avro
No Format docker run -it ruby:2.3 gem install avro:X.Y.Z
Javascript - https://www.npmjs.com/package/avro-js
No Format docker run -it node:10 npm install avro-js@X.Y.Z
C# - https://www.nuget.org/packages/Apache.Avro / https://www.nuget.org/packages/Apache.Avro.Tools
No Format docker run -it mcr.microsoft.com/dotnet/core/sdk:3.1 $ dotnet new console -o test-project && cd test-project && dotnet add package Apache.Avro --version X.Y.Z $ dotnet tool install --global Apache.Avro.Tools --version X.Y.Z
Perl - https://metacpan.org/release/Avro
No Format docker run -it perl:5 cpan Avro