You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

This page describes how to make a release of Apache Bigtop (incubating). Most of the content is shamelessly stolen from the the Whirr release process – thanks Whirr team!

0. Before you begin.

Apache Bigtop (incubating) is following a predictable "train model" of releases popularized by the Ubuntu Linux. Currently releases happen every quarter. A release of Bigtop is a culmination of a quarter of development activities and also a testament to the particular state of the Hadoop ecosystem at the time. IOW, what are the most cutting edge versions of the components that still work together as an integrated system.

All release activities are coordinated by a volunteer release manager (RM) who is chosen during the release planning. If you happen to be a first time RM here's what you need to do in order to be able to carry out the mechnics of the release

  1. Generate PGP code signing keys
  2. Ensure that your PGP signing keys are available in: https://svn.apache.org/repos/asf/incubator/bigtop/dist/KEYS More details can be found here.
  3. Copy the new KEYS file to the release folder /www/www.apache.org/dist/incubator/bigtop on people.apache.org

If you are not already a member of the Web Of Trust (WOT) it would be a good idea to do so. (contact one of the prior release managers, e.g. Tom, Patrick, etc...). You can read more about key signing here.

You need to be a member of the "incubator" unix group on people.apache.org. Ask for help on the incubator general@ list. Here's an example http://bit.ly/9Wkdzg

Ensure that you have setup your ssh keys on people.apache.org, otherwise you'll have to enter your login password a number of times (best use ssh-agent for this as well). A good overview of this process can be found here (ssh-copy-id and ssh-agent in particular)

Done with the above? Proceed with the rest of the steps:

1. Scrubbing the JIRA.

3-4 weeks before a committed release date a release manager (RM) for a given release is supposed to start a process of scrubbing the JIRAs. This involves:

  1. taking care of all the patch available issues for a given release
  2. send an email to the bigtop-dev mailing list reminding the community of the upcoming release and asking them to spend 2-3 days raising the priority of the "must fix" issues for a given release to a "Blocker" status AND making sure that Fix Versions field is set to an upcoming release. Of course, it goes without saying that people have to volunteer to work on this issues – this is not an exercise of assigning work to others. Typically everybody has their own favorite issues that they would like to see fixed in the upcoming release, however do encourage folks to also take a look at the Unscheduled issues since those tend to accumulate lurkers. Also 'git grep' for the 'FIXME/WORKAROUD' prefix in the source code as to identify anything that doesn't need to be worked around anymore
  3. after waiting for 2-3 days for community to respond and for the Blockers to settle down – make sure that the resulting list looks reasonable and that there is a general expectation that the release can happen given the state of the source base.

2. Create a Release Series Branch

This only needs doing if this is the first release in a series (X.Y.0).

  • Update CHANGES.txt in trunk to replace Trunk (unreleased changes) with Release X.Y.0 - YYYY-MM-DD. Commit: 
    svn commit -m "Preparing for release X.Y.Z"
  • Create a branch for the release series: 
    svn copy https://svn.apache.org/repos/asf/incubator/bigtop/trunk \
https://svn.apache.org/repos/asf/incubator/bigtop/branches/branch-X.Y -m "Branching for X.Y releases"
  • Add back Trunk (unreleased changes) to CHANGES.txt in trunk.
  • Bump the version number in trunk: 
    for file in $(find . -name pom.xml); do
  sed -i -e "s/0.X.0-incubating-SNAPSHOT/0.Y.0-incubating-SNAPSHOT/" $file;
done
  • Commit these changes to trunk.
  • Checkout the release branch 
    svn checkout https://svn.apache.org/repos/asf/incubator/bigtop/branches/branch-X.Y
  • Update the release branch's version information: the version number in the release branch ends in -SNAPSHOT, but we need to remove this for the release. For example, 0.1.0-SNAPSHOT needs to be changed to 0.1.0. 
    for file in $(find . -name pom.xml); do
  sed -e "s/0.X.0-SNAPSHOT/0.X.0/" $file;
done

3. Build and run Package and Smoke Tests

TODO

4. Generate the Release Notes

JIRA has the ability to generate release notes automatically (this is why it is so important to keep the fix version number accurate).

https://issues.apache.org/jira/secure/ConfigureReleaseNote.jspa?projectId=12311420

Manually check this list for accuracy! I've repeatedly seen closed bugs that were not fixed (i.e., duplicate) marked with a fix version, so that they incorrectly show up in this list. Find those, edit them to remove the fix release (only actually fixed bugs should have a fix release) and re-run the report.

Select the correct version. Ask for both plain text and HTML formats.

Paste the plain text notes to the top of CHANGES.txt. Paste the HTML version into src/site/xdoc/release-notes.xml. Check this into trunk and the branch.

Wrap the title ("Release Notes - Whirr - Version X.Y.Z") inside an <h1> element.

5. Build and Deploy Artifacts

Create a maven settings file ~/.m2/settings.xml with the following content:

<settings>
<servers>
<server>
     <id>apache.snapshots.https</id>
     <username>APACHE-ID</username>
     <password>APACHE-PASSWORD</password>
   </server>
   <server>
     <id>apache.staging.https</id>
     <username>APACHE-ID</username>
     <password>APACHE-PASSWORD</password>
   </server>
   <server>
     <id>apache.releases.https</id>
     <username>APACHE-ID</username>
     <password>APACHE-PASSWORD</password>
   </server>
</servers>
</settings>

Build the artifacts:

mvn site
mvn -Prelease package assembly:assembly

The following command deploys the artifacts, checksums, and signatures (you will need to enter a GPG passphrase) to the Apache Staging repo.

mvn deploy -Prelease -Ppackage -Pdeploy -Pjavadoc

If this step fails with an Access denied error check that you have the required permissions on Nexus.

Login to https://repository.apache.org using your Apache SVN credentials. Click on Staging on the left. Then click on org.apache.whirr in the list of repositories. In the panel below you should see an open repository that is linked to your username and IP. Select this repository and click Close. This will close the repository from future deployments and make it available for others to view.

6. Copy Release Artifacts

The artifacts that end up in the distribution directory are the source distributions (along with their checksums and signatures), so they need to be copied from the Maven repo to the release directory on people.apache.org so folks can vote on them:

ssh people.apache.org
VERSION=X.Y.Z
mkdir ~/public_html/bigtop-$VERSION-RC0
cd ~/public_html/bigtop-$VERSION-RC0
for ext in "" .asc .md5 .sha1; do
  wget --no-check-certificate https://repository.apache.org/content/repositories/orgapachebigtop-[YOUR REPOSITORY ID]/org/apache/bigtop/bigtop/$VERSION/bigtop-$VERSION-src.tar.gz$ext
done

7. Sanity Check

TODO

  • Check the MD5 checksums

8. Commit and Tag

Commit changes to subversion, and tag:

svn commit -m "Preparing for release X.Y.Z"
svn copy https://svn.apache.org/repos/asf/incubator/bigtop/branches/branch-X.Y https://svn.apache.org/repos/asf/incubator/bigtop/tags/release-X.Y.Z -m "Bigtop X.Y.Z release."

9. Run the Vote

Run the vote on the dev-bigtop@incubator.apache.org

Here is an example email:

To: "Bigtop Developers List" <dev-bigtop@incubator.apache.org>
Subject: [VOTE] Release Bigtop version X.Y.Z

This is the first release for Apache Bigtop (incubating), version X.Y.Z. 

It fixes the following issues:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12318889

*** Please download, test and vote by [3 working days after sending].

Note that we are voting upon the source (tag)

Source and binary files:
http://people.apache.org/~[YOUR USERID]/bigtop-X.Y.Z-RC0

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachebigtop-[YOUR REPOSITORY ID]

The tag to be voted upon:
http://svn.apache.org/repos/asf/incubator/bigtop/tags/release-X.Y.Z

Whirr's KEYS file containing PGP keys we use to sign the release:
http://svn.apache.org/repos/asf/incubator/bigtop/dist/KEYS

The release needs 3 +1 votes from the IPMC.

10. Roll Out

Assuming the vote passes, the release can be rolled out as follows:

Move Artifacts into Place

This step makes the artifacts available on the mirrors.

ssh people.apache.org
VERSION=X.Y.Z
CANDIDATE=C
cp -r ~/public_html/bigtop-$VERSION-RC$CANDIDATE /www/www.apache.org/dist/incubator/bigtop/bigtop-$VERSION
cd /www/www.apache.org/dist/incubator/bigtop
chgrp -R incubator bigtop-$VERSION
chmod -R g+w bigtop-$VERSION
rm stable
ln -s bigtop-$VERSIONstable
rm -r /www/www.apache.org/dist/incubator/bigtop/bigtop-$PREVIOUS_VERSION # remove older versions

The last line is to remove the previous version, since only the most recent version on a particular branch should be in the dist directory (older versions are archived automatically, see http://archive.apache.org/dist/incubator/bigtop/ and http://www.apache.org/dev/mirrors.html).

Log in to https://repository.apache.org, click on Staging on the left. Select the repository that you closed earlier, and click Release, using a description like "Apache Bigtop X.Y.Z artifacts". This will make the artifacts publicly available.

Wait 24 Hours

It takes up to 24 hours for all the mirrors to sync, so don't announce the new release just yet.

Build and Deploy Site

mvn site-deploy

This will deploy the generated website to /www/bigtop.apache.org/ on people.apache.org. You can SSH there to check that things look OK.

It will take an hour or so for the website to become live, but you can inspect the website by temporarily setting your browser proxy to see the new, unmirrored content (I used FoxyProxy in Firefox) as described in http://www.apache.org/dev/project-site.html.

11. Announce the Release

TODO: Add a news section to the website.

Send an email to announce@apache.org (the from: address must be @apache.org). E.g.

To: announce@apache.org, bigtop-user@incubator.apache.org, bigtop-dev@incubator.apache.org
Subject: [ANNOUNCE] Apache Bigtop X.Y.Z released

The Apache Bigtop (incubating) team is pleased to announce the release of Bigtop X.Y.Z.

The release is available here:
http://www.apache.org/dyn/closer.cgi/bigtop/

The full change log is available here:
https://issues.apache.org/jira/browse/BIGTOP/fixforversion/12315111

We welcome your help and feedback. For more information on how to
report problems, and to get involved, visit the project website at
http://incubator.apache.org/bigtop

The Apache Bigtop Team

12. Add the Next Release to JIRA

Add the next version number (e.g. 0.2.0 after 0.1.0) to JIRA using this
link: https://issues.apache.org/jira/secure/project/ManageVersions.jspa?pid=12311420.

In JIRA mark the released version as "released" on the "manage versions" page. Be sure to fill in a date if not already specified.

  • No labels