DRAFT: How to Release

Introduction

From time to time, Impala creates releases. These are tarballs of the Impala source that follow the rules below.

Decisions about releases are made by four groups:

A Release Manager, who must be a committer, does the work of creating the release, signing it, counting votes, and so on.
The community, who share in the discussion of whether it is the right time to create a release and what that release should contain. The community can also cast non-binding votes on the release.
The PMC, who cast binding votes on the release.
The Incubator PMC, who must approve all Impala releases until Impala graduates to an Apache Top Level Project.

This page describes the procedures that the release manager and voting PMC members take during release time.

How to Manage a Release

Release managers are self-selected. Release management must happen partially on "hardware owned and controlled by the committer". In particular, release managers are not permitted to store code signing keys on hardware they do not own or control.

Make a code signing key if you don't have one yet
1. Publish your key if you haven't done so yet
If you haven't done so yet, add yourself to the Apache Web-of-Trust by meeting face-to-face with a person so they can sign your key
1. Publish your signed key
If you haven't done so before, add your key to the KEYS file in the release staging area and the release distribution area
1. The staging area is https://dist.apache.org/repos/dist/dev/incubator/impala/. Once graduated, it will presumably be https://dist.apache.org/repos/dist/dev/impala/
2. The release area is https://dist.apache.org/repos/dist/release/incubator/impala/, and will presumably be https://dist.apache.org/repos/dist/release/impala/ after graduation.
Pick a commit in the "master" branch you want to release from and test it.
Test the licencing using Apache RAT; follow the instructions in bin/check-rat-report.py.
Propose a release on the dev@ list. It should start with the string "[DISCUSS]". Explain that this is not a "[VOTE]", and that anyone may participate.
Make a new branch off of your commit called "branch-x.y.z", where x.y.z is the version you want to release
1. In this branch (but not in master) update the version number in bin/save-version.sh to x.y.z-RELEASE

Continue testing. If you find bugs, file them. When they are fixed, cherry-pick the fixes from master to your branch that you want to include in the release

git fetch apache # to make sure you have the latest master
git log apache/master # to find the commits you want to cherry-pick
git checkout apache/branch-x.y.z
git checkout -b x.y.z-patch-foo # the name doesn't matter - it will only be in your local workstation
git cherry-pick b4d1a3... # or whatever git hashes
# then resolve conflicts, if there are any
# cherry-pick some more commits:
git cherry-pick ....
git cherry-pick .....
git push apache HEAD:branch-x.y.z

At that time, tag the tree at the release candidate

git fetch apache
git checkout apache/branch-x.y.z
git tag -a x.y.z-rcw # when making release candidate w of version x.y.z
git push apache x.y.z-rcw

Make a release tarball:

Make the tarball using git archive. Name it apache-impala-incubating-x.y.z.tar.gz, or apache-impala-x.y.z.tar.gz if Impala has graduated from the incubator.
```
git archive --prefix=apache-impala-incubating-x.y.z/ -o /tmp/apache-impala-incubating-x.y.z.tar.gz x.y.z-rcw
```

Make signature, as well as md5 and sha1 checksums:

# Make the signature:
gpg -u YOUR_APACHE_USER_NAME@apache.org --armor --output apache-impala-incubating-x.y.z.tar.gz.asc --detach-sign apache-impala-incubating-x.y.z.tar.gz
 
# Make sure it worked:
gpg --verify apache-impala-incubating-x.y.z.tar.gz.asc
 
# Make checksums:
md5sum apache-impala-incubating-x.y.z.tar.gz > apache-impala-incubating-x.y.z.tar.gz.md5
sha1sum apache-impala-incubating-x.y.z.tar.gz > apache-impala-incubating-x.y.z.tar.gz.sha
 
# Make sure they worked:
md5sum --check apache-impala-incubating-x.y.z.tar.gz.md5
sha1sum --check apache-impala-incubating-x.y.z.tar.gz.sha

Before uploading your release candidate, follow the procedure in the section below on how to vote on a release. Don't upload until you could vote +1.
Get commitments from at least five PMC members to vote on the artifact in the time frame for the upcoming vote. While we are incubating, the list of PMC members is at http://incubator.apache.org/projects/impala.html. Once Impala graduates, that list will presumably be available at http://people.apache.org/committers-by-project.html#impala-pmc.

Upload the artifacts. While incubating, the location is https://dist.apache.org/repos/dist/dev/incubator/impala/. Once graduated, it will presumably be https://dist.apache.org/repos/dist/dev/impala/. Upload all four artifacts. Do not overwrite old release candidates.

svn checkout https://dist.apache.org/repos/dist/dev/incubator/impala/
cd impala/
# The directory layout is x.y.z/RCw, where w is the release candidate number - RC1 is the first candidate, RC2 the second, and so on.
mkdir -p x.y.z/RCw
cd x.y.z/RCw
cp /tmp/apache-impala-incubating-x.y.z.tar.gz* ./
svn add .
svn commit --username=YOUR_APACHE_USER_NAME -m "Impala x.y.z release candidate w"

Take a vote on dev@. Your vote email should:
1. Have a subject that starts with "[VOTE]"
2. Explain what the vote is about
3. Explain how to find the artifacts for testing, and include the tag and git tree hash they correspond to
4. Explain what each type of vote means
5. Explain the conditions for the vote passing, including how long the vote will remain open for.
6. Include a link to this wiki page so voters can read the instructions in it on how to test, verify, and vote.
7. Be consistent with the Impala bylaws. For instance, at the moment I am writing this wiki page, votes stay open for 72 hours.
When the vote closes, tally up the votes (who votes what) in a thread with the same title as the vote thread, but with "[RESULT] " prepended. That email should include a list of every vote, ala:
```
+1 (binding):
 
Alice Bobopolis
Carol Davestein
 
-1 (non-binding):
 
Emily Frankfurt
```
If the vote passes, and Impala has yet to graduate, take a vote in the incubator PMC, following current incubator policy.
1. Subscribe to general@incubator.apache.org by sending mail to general-subscribe@ and responding to the email it sends back.
2. Send an email to that list including:
  1. A link to the archive of the proposal thread from the dev@impala mailing list, available either http://mail-archives.apache.org/mod_mbox/impala-dev/ or https://lists.apache.org/list.html?dev@impala.apache.org
  2. A link to the archive of the [RESULT] of the vote from the dev@impala mailing list.
  3. A link to the artifacts for testing
  4. A link to the KEYS file with the signatures
  5. A link to the git tag the release candidate tarball was created from
  6. The hash of the git commit the release candidate tarball was created from
  7. ```
  This vote will be open for at least 72 hours, or until the necessary
  number of votes (3 +1) is reached.
  
  [ ] +1 Approve the release
  [ ] -1 Don't approve the release (please provide specific comments)
```

Once that vote passes, tag the git tree at the release:

git fetch apache
git checkout apache/branch-x.y.z
git tag -a x.y.z
git push apache x.y.z

Publish the release. While incubating, the location is https://dist.apache.org/repos/dist/release/incubator/impala/. Once graduated, it will presumably be https://dist.apache.org/repos/dist/release/impala/. Upload all four artifacts.

svn checkout https://dist.apache.org/repos/dist/release/incubator/impala/
cd impala/
mkdir x.y.z
cd x.y.z
cp /tmp/apache-impala-incubating-x.y.z.tar.gz* ./
svn add .
svn commit --username=YOUR_APACHE_USER_NAME
# The commit message, should look like (without the '#'s):
#
# Impala release x.y.z
# 
# PPMC vote results thread: ...
#
# IMPC vote results thread: ...

Wait 24 hours for mirrors to catch up.
Announce the release to dev@, user@, and
Send a patch review to the master branch to update its version number to "p.q.r-SNAPSHOT" (where p.q.r is greater than x.y.z)

How to Vote on a Release Candidate

Download the tarball. Check the signature and the checksums.

# change to a new directory
cd $(mktemp -d)
 
# Download the keys of the release managers
wget https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
gpg --import KEYS
 
# Set the keys of the release managers as trusted
gpg --edit-key APACHE_USER_NAME_OF_RELEASE_MANAGER trust
# At the prompt, enter '5' for "I trust ultimately", then 'y' for "yes", then 'q' for "quit"
 
# Download the release artifacts:
wget https://dist.apache.org/repos/dist/dev/incubator/impala/x.y.z/RCw/apache-impala-incubating-x.y.z.tar.{gz,gz.asc,gz.md5,gz.sha}
 
# Check the checksums:
md5sum --check apache-impala-incubating-x.y.z.tar.gz.md5
sha1sum --check apache-impala-incubating-x.y.z.tar.gz.sha

# Check the signature:
gpg --verify apache-impala-incubating-x.y.z.tar.gz.asc

Check that it matches the upstream tag

# download the repo
pushd $(mktemp -d)
git clone https://git-wip-us.apache.org/repos/asf/incubator-impala.git
cd *
git checkout x.y.z-rcw
export IMPALA_GIT=$(pwd)
popd 
 
# compare the tarball and the repo:
tar xzf apache-impala-incubating-x.y.z.tar.gz
diff -r apache-impala-incubating-x.y.z "${IMPALA_GIT}"
# You should see something like "Only in incubator-impala: .git", but no other output

Test the release quality, possibly using bin/run-all-tests.py. The ASF requires in its "Release Policy" that: "Before voting +1 PMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases."
Check compliance with ASF release policy. Use Apache RAT and follow the instructions in bin/check-rat-report.py to check licence compliance.
If it is an official "[VOTE]" thread, vote +1 or -1. If you are a PMC member, add "(binding)" after your vote; otherwise, add "(non-binding)". If you vote -1, explain why.

Space shortcuts

Page tree

Introduction

How to Manage a Release

How to Vote on a Release Candidate