Prerequisites
Before starting the process, you need to ensure several things before proceed to create a release.
- You need to clone the latest pinot code base in your local machine
- You need to ensure that you have the access to Apache Repository Manager. Try to log in with Apache LDAP credential.
You need to check out Apache staging & release distribution directories for Pinot.
svn co https://dist.apache.org/repos/dist/release/incubator/pinot pinot-release-dist svn co https://dist.apache.org/repos/dist/dev/incubator/pinot pinot-dev-dist
https://dist.apache.org/repos/dist/release/incubator/pinot: this is where the actual release (PMC approved) will be placed. Do not upload anything here until voting from both dev@pinot and general@incubator pass.
- https://dist.apache.org/repos/dist/dev/incubator/pinot: this is where we put the release candidates and link for this will be used for reference during the voting process.
GPG Key Setup
The release manager must hold a GPG key for signing the release artifacts. GPG key has to be associated with the apache email address. One can create the GPG by using the following cmd. Note that the key length should be 4096 bits. Please refer to the example below:
$ gpg2 --full-generate-key gpg (GnuPG) 2.1.22; Copyright (C) 2017 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? 1 RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) 4096 Please specify how long the key should be valid. 0 = key does not expire <n> = key expires in n days <n>w = key expires in n weeks <n>m = key expires in n months <n>y = key expires in n years Key is valid for? (0) Key does not expire at all Is this correct? (y/N) y ... gpg: key 2C9B34D6A54AAFAE marked as ultimately trusted ...
After you generate the GPG key, copy the key. "e.g. gpg: key 2C9B34D6A54AAFAE marked as ultimately trusted" (From here, I will use GPG_KEY to refer the key).
$ GPG_KEY=<your_gpg_key> # fill in your key
Once you generate the gpg key, you need to go through the following steps:
Upload GPG public key to a public key server. Copy the following command's output to "Submit a key" box and click the submit button. http://pgp.mit.edu
$ gpg --export --armor $GPG_KEY -----BEGIN PGP PUBLIC KEY BLOCK----- ... -----END PGP PUBLIC KEY BLOCK-----
Update key fingerprint at https://id.apache.org
$ gpg --fingerprint $GPG_KEY
Copy the hex numbers (e.g. "0D9E 9997 4D5A 59F1 7183 362F 2C9B 34D6 A54A AFAE") to "OpenPGP Public Key Primary Fingerprint" box and submit the change.
Update KEYS file under pinot-release-dist
$ cd pinot-release-dist # Append the public key to KEYS file gpg --list-sigs $GPG_KEY && gpg --armor --export $GPG_KEY >> KEYS... # Commit the change svn commit -m "Update Pinot KEYS file for $GPG_KEY"
After following 1-3, you should have the following:
- Your public key is listed in https://dist.apache.org/repos/dist/release/incubator/pinot/KEYS
- Your public key is viewable at https://people.apache.org/keys/committer/<apache_ldap_id>.asc (e.g. https://people.apache.org/keys/committer/snlee.asc)
- Your public key is viewable at https://people.apache.org/keys/group/pinot.asc
Maven Configuration Setup
As part of the release process, we will upload our artifacts to maven central staging repository and "maven-release-plugin" will do it for you automatically for the most of part. In order to use the plugin, we need to set up the maven config correctly.
$ vim ~/.m2/settings.xml
# Paste the below text
<settings>
<servers>
<server>
<id>apache.snapshots.https</id>
<username>{apache LDAP id}</username>
<password>{apache user password}</password>
</server>
<server>
<id>apache.releases.https</id>
<username>{apache LDAP id}</username>
<password>{apache user password}</password>
</server>
<server>
<id>gpg.passphrase</id>
<passphrase>{gpg key phrase}</passphrase>
</server>
</servers>
<profiles>
<profile>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<gpg.keyname>{GPG_KEY}</gpg.keyname>
</properties>
</profile>
</profiles>
</settings>
Also, We recommend to add the following line to your `~/.bashrc` file.
$ vim ~/.bashrc # add the following line export GPG_TTY=$(tty) $ source ~/.bashrc
Adding above line make the terminal UI correctly show when you need to enter the key phrase for your key when signing the artifacts.
Creating a Release
Before we move on to the voting, we need to prepare the release along with the below information:
- GPG Key that the artifacts are signed with - From GPG Key Setup
- Git tag - Step 2
- Git hash for the release - Step 2 (Pick the git hash from the tag created during step 2)
- Release Note - Step 3
- Staging Repository link (Nexus) - Step 4
- Release Candidate under https://dist.apache.org/repos/dist/dev/incubator/pinot - Step 5
After following the steps below, you should have all information listed above.
Prepare a release branch
$ git checkout -b release-<version>-rc<rc_num> $ git push origin release-<version>-rc<rc_num> # Example $ git checkout -b release-0.1.0-rc0 $ git push origin release-0.1.0-rc0
Prepare the release (Please refer "Preparing a release" section in http://plc4x.apache.org/developers/release.html to see what actually happens with the command below - has a good explanation along with screenshots, our gitbox url for checking is https://gitbox.apache.org/repos/asf?p=incubator-pinot.git)
$ mvn release:prepare
- Prepare the release note. Step 2 will generate the new tag and it will automatically show up on https://github.com/apache/incubator-pinot/releases
- Click "Draft a new release" button, put "tag version" as the one you used from the previous step (e.g. release-0.1.0-rc0).
- Fill in the release note and publish the release. NOTE: you need to check "This is a pre-release" box in order not to violate Apache terms.
Perform the release (publishing artifacts to "staging repository")
$ mvn release:perform
After run this, log in to Nexus (https://repository.apache.org) and select Stating Repositories and find the repository with the name "orgapachepinot-{number}". Select that and click the "Close" button. This will trigger to Nexus to do some checks on artifacts (checksum, signatures..etc). Check https://repository.apache.org/content/repositories/orgapachepinot-{number} shows correctly.
Create a source/binary tarballs & Staging source and binary release tarballs
$ cd <pinot_source_code_root_path># Check out the release candidate tag$ git checkout tags/release-<version>-rc<rc_num># Check git hash for the official release $ git log# Create the package $ mvn install -DskipTests -Papache-release,bin-dist $ cd pinot-distribution/target $ ls ... -rw-r--r-- 1 snlee eng 115915316 Mar 21 14:25 apache-pinot-incubating-0.1.0-bin.tar.gz -rw-r--r-- 1 snlee eng 490 Mar 21 14:25 apache-pinot-incubating-0.1.0-bin.tar.gz.asc -rw-r--r-- 1 snlee eng 128 Mar 21 14:26 apache-pinot-incubating-0.1.0-bin.tar.gz.sha512 -rw-r--r-- 1 snlee eng 37404419 Mar 21 14:25 apache-pinot-incubating-0.1.0-src.tar.gz -rw-r--r-- 1 snlee eng 490 Mar 21 14:25 apache-pinot-incubating-0.1.0-src.tar.gz.asc -rw-r--r-- 1 snlee eng 128 Mar 21 14:26 apache-pinot-incubating-0.1.0-src.tar.gz.sha512 ... # Copy files to the pinot-dev-dist $ mkdir /path/to/pinot-dev-dist/apache-pinot-incubating-<version>-rc<rc_num> $ cp apache-pinot-incubating-0.1.0-SNAPSHOT-*.tar.gz* /pat/to/pinot-dev-dist/apache-pinot-incubating-<version>-rc<rc_num> # Commit the files to staging svn repository $ cd /path/to/pinot-dev-dist $ svn add apache-pinot-incubating-<version>-rc<rc_num> $ svn commit -m "Update apache-pinot-incubating-<version>-rc<rc_num>"
After the above step, check https://dist.apache.org/repos/dist/dev/incubator/pinot/ to see if the files are uploaded correctly. requires -src/bin.tar.gz, tar.gz.asc, tar.gz.sha512 files.
Once you have all 6 information (gpg key, git tag, git hash, release note, staging repository, staged tarballs), release is now ready for voting process!
Call the Vote
As a first step, we need to go through the voting at dev@pinot mailing list. Send out the mail for voting. The example is present at the following links. The voting should be open for at least 72 hours and it requires minimum 3 +1 vote. Once the voting passes, you can send out the result mail.
dev@pinot vote https://lists.apache.org/thread.html/f136d3eaa9dfbab6e17b262a5542813099f2b128465d9d17ef69defd@%3Cdev.pinot.apache.org%3E
dev@pinot vote result https://lists.apache.org/thread.html/ce58034678349f82afc5f8ed2edd435875301183554f964778dffb7a@%3Cdev.pinot.apache.org%3E
Once the vote is passed from Pinot community, we can open the vote in general@incubating channel. The process & requirements are similar (3 +1 vote). Once the voting passes, you can close the voting by sending a voting result. The examples are shown at the following.
general@incubator vote https://lists.apache.org/thread.html/1e1e57cdd5361371f813b67f8caa0de9bc6bf81559bf4cd0b6de7dba@%3Cgeneral.incubator.apache.org%3E
general@incubator vote result https://lists.apache.org/thread.html/7255544ff38ea30d364ef49a2f3ee0a731e5fd26df48ddf63f63ba25@%3Cgeneral.incubator.apache.org%3E
Once the voting passes, it is now officially approved by Apache and we just need to do some final steps.
Finalize a Release
Move staging tarballs to the svn directory for official release
$ svn mv https://dist.apache.org/repos/dist/dev/incubator/pinot/apache-pinot-incubating-<version>-rc<rc_num> https://dist.apache.org/repos/dist/release/incubator/pinot/apache-pinot-incubating-<version>
Check https://dist.apache.org/repos/dist/release/incubator/pinot
After several hours later, check http://www.apache.org/dyn/closer.cgi/incubator/pinot/apache-pinot-incubating-<version> to see if it shows up.
(e.g. http://www.apache.org/dyn/closer.cgi/incubator/pinot/apache-pinot-incubating-0.1.0)Update git tag for official release
$ git checkout tags/release-<version>-rc<rc_num>$ git tag release-<version> -m "Apache Pinot (Incubating) <version>" $ git push origin release-<version>
Update Release note for the official release on Github https://github.com/apache/incubator-pinot/releases (Draft a new release, this time you don't need to check "pre-release" box)
# In case you want to clean up the rc tag to not to show on the Github release page, $ git tag -d release-<version>-rc<rc_num>$ git push origin :refs/tags/release-<version>-rc<rc_num>
- Publish the maven artifacts. Log in to Nexus, select the staging repository (orgapachepinot-<number>) and click Release button. Several hours later, the new version should show up at https://repo.maven.apache.org/maven2/org/apache/pinot
Announce to the world
Once you finalized the release, you now need to announce this to the world.
- Send the mail to dev@pinot, general@incubator about the release announcement
dev@pinot https://lists.apache.org/thread.html/1b58323b12666bdd855e66cdb3cd72462ab2e86b7cba32757bdf7393@%3Cdev.pinot.apache.org%3E
general@incubator https://lists.apache.org/thread.html/3c1778095a992f8df08bf4774c1e1aea8e0e1914bdab02a90f582ac4@%3Cgeneral.incubator.apache.org%3E - Update the Announcements on website
- Update https://pinot.apache.org/download page. Example pr: https://github.com/apache/incubator-pinot-site/pull/7
Reference
- http://plc4x.apache.org/developers/release.html
- https://cwiki.apache.org/confluence/display/GOBBLIN/Release+Process