Before starting the process, you need to ensure a few things are set up correctly.

Clone Pinot code base

Creating the key

The GPG key has to be associated with the apache email address. One can create the GPG by using the following cmd. Note that the key length should be 4096 bits. See the example belowClone the latest pinot code base in your local machine. Create a branch for your release from the point at which you wish to cut a release. his should be a commit hash that has been tested and verified in reasonable settings (e.g Pinot software has been tested to run in production with various use cases at a particular commit hash). For example:

Code Block

language	bash

$ gitgpg2 checkout -b release-<version> <commit-hash>
$ git push origin release-<version>

# Example
$ git checkout -b release-0.3.0-rc 9abf0a4b105d624ec310273c2bd7f52c1f3ce12a
$ git push origin release-0.3.0-rc

Run the test with sampling OFF, with the following changes (do not check in)

Code Block

language	bash

--- a/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/BaseClusterIntegrationTestSet.java
+++ b/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/BaseClusterIntegrationTestSet.java
@@ -297,10 +297,6 @@ public abstract class BaseClusterIntegrationTestSet extends BaseClusterIntegrati
     int maxNumQueriesToSkipInQueryFile = getMaxNumQueriesToSkipInQueryFile();
     try (BufferedReader reader = new BufferedReader(new FileReader(queryFile))) {
       while (true) {
-        int numQueriesSkipped = RANDOM.nextInt(maxNumQueriesToSkipInQueryFile);
-        for (int i = 0; i < numQueriesSkipped; i++) {
-          reader.readLine();
-        }
 
         String queryString = reader.readLine();
         // Reach end of file.
@@ -335,10 +331,6 @@ public abstract class BaseClusterIntegrationTestSet extends BaseClusterIntegrati
     try (BufferedReader reader = new BufferedReader(new FileReader(queryFile))) {
       while (true) {
         int numQueriesSkipped = RANDOM.nextInt(maxNumQueriesToSkipInQueryFile);
-        for (int i = 0; i < numQueriesSkipped; i++) {
-          reader.readLine();
-          queryId++;
-        }

Run the test, and make sure there are no failures (this can take 5-6 hours)

Code Block

language	bash

mvn clean test

Note
The update of LICENSE, NOTICE files, and the run of `mvn release:prepare` command, `mvn release:commit` command, etc. should be done on this branch, NOT on the master.

Prepare the release notes

Note
Please do not gather all the commit messages and turn that into release notes.

You need to prepare good release notes that highlights:

Important features added in the new release.
Any deprecated configuration or external methods, noting these these may be removed later.
Any configuration/methods that was deprecated earlier, and have been removed from this release.
Any special notes for upgrading from previous releases (e.g. for zero-downtime upgrade, please upgrade to release X before this one).
Other special notes

Here is a command you can use to get all the check-ins since the last release

Code Block

language	bash

git log 2a3860229^.. -- pinot-api -- pinot-azure-filesystem -- pinot-broker -- pinot-common -- pinot-connectors -- pinot-controller -- pinot-core -- pinot-hadoop -- pinot-hadoop-filesystem -- pinot-minion -- pinot-orc -- pinot-parquet -- pinot-server -- pinot-tools -- pinot-transport
# Assuming 2a3860229 was the commit hash in the master where the previous release was branched off
# Each argument following -- is a directory under pinot source tree (note that we have excluded the 'docs' directory

Once you have the commits, you are ready to come up with good material for release notes.

Draft the release notes in a text file (in markdown syntax) and keep it ready. You will need it in a later step.

Check out staging and release distributions

You need to check out Apache staging & release distribution directories for Pinot.

Code Block

language	bash

svn co https://dist.apache.org/repos/dist/release/incubator/pinot pinot-release-dist
svn co https://dist.apache.org/repos/dist/dev/incubator/pinot pinot-dev-dist

Info

Final PMC approved release will be placed in https://dist.apache.org/repos/dist/release/incubator/pinot. Do NOT upload anything here until voting from both dev@pinot and general@incubator pass.

Release candidates will be in https://dist.apache.org/repos/dist/dev/incubator/pinot. This link will be used for voting process.

Ensure access to Apache repository manager

You need to ensure that you have the access to Apache Repository Manager (Nexus). Try to log in with Apache user name and password.

Create PGP key and upload it to public servers

Note

It make take a few iterations (each time waiting for a day) to get this right, so we suggest that you get this started as early as possible in the release process, while you work on other steps (like updating LICENSE and NOTICE files).

If you have already done this for your apache user ID, you can skip this part.

You need to create a PGP key and upload it to public servers. Apache sites pick up the certificate from public repositories once a day. We strongly suggest that you read through this page first: https://www.apache.org/dev/release-signing.html. Here are the steps involved.

Creating the key

The GPG key has to be associated with the apache email address. One can create the GPG by using the following cmd. Note that the key length should be 4096 bits. See the example below:

Code Block

language	bash

$ gpg2 --full-generate-key
gpg (GnuPG) 2.1.22; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 1

RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096

Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0)

Key does not expire at all
Is this correct? (y/N) y

...

gpg: key 2C9B34D6A54AAFAE marked as ultimately trusted
...

After you generate the GPG key, copy the key into an environment variable. Look for the line in the output that says "marked as ultimately trusted". For example, in the above output: "gpg: key 2C9B34D6A54AAFAE marked as ultimately trusted" (From here, I will use GPG_KEY to refer the key).

Code Block

language	bash

$ GPG_KEY=<your_gpg_key>  # fill in your key from the "ultimately trusted" line as above

Update the key preference, if necessary

Make sure the key has a preference of SHA512 (otherwise, the key does not show up in apache site). See https://www.apache.org/dev/openpgp.html#key-gen-avoid-sha1

Uploading the generated key

Upload GPG public key to a public key server. Copy the following command's output (between the lines that Begin and end certificate, including the BEGIN and END line) to "Submit a key" box and click the submit button. http://pgp.mit.edu. You can also upload it to other sites (See https://www.apache.org/dev/release-signing.html#keyserver for some servers, and also a command-line way to send key to public servers)

Code Block

language	bash

$ gpg --export --armor $GPG_KEY

-----BEGIN PGP PUBLIC KEY BLOCK-----
...
-----END PGP PUBLIC KEY BLOCK-----

Update key fingerprint at https://id.apache.org. Type the command given below to display the fingerprint of the key.

Code Block

language	bash

$ gpg --fingerprint $GPG_KEY

...

--full-generate-key
gpg (GnuPG) 2.1.22; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 1

RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096

Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0)

Key does not expire at all
Is this correct? (y/N) y

...

gpg: key 2C9B34D6A54AAFAE marked as ultimately trusted
...

After you generate the GPG key, copy the key into an environment variable. Look for the line in the output that says "marked as ultimately trusted". For example, in the above output: "gpg: key 2C9B34D6A54AAFAE marked as ultimately trusted" (From here, I will use GPG_KEY to refer the key).

Code Block

language	bash

$ GPG_KEY=<your_gpg_key>  # fill in your key from the "ultimately trusted" line as above

Update the key preference, if necessary

Make sure the key has a preference of SHA512 (otherwise, the key does not show up in apache site). See https://www.apache.org/dev/openpgp.html#key-gen-avoid-sha1

Uploading the generated key

Upload GPG public key to a public key server. Copy the following command's output (between the lines that Begin and end certificate, including the BEGIN and END line) to "Submit a key" box and click the submit button. http://pgp.mit.edu. You can also upload it to other sites (See https://www.apache.org/dev/release-signing.html#keyserver for some servers, and also a command-line way to send key to public servers)
Code Block
language bash
$ gpg --export --armor $GPG_KEY -----BEGIN PGP PUBLIC KEY BLOCK----- ... -----END PGP PUBLIC KEY BLOCK-----
Update key fingerprint at https://id.apache.org. Type the command given below to display the fingerprint of the key.
Code Block
language bash
$ gpg --fingerprint $GPG_KEY
Copy the hex numbers (e.g. "0D9E 9997 4D5A 59F1 7183 362F 2C9B 34D6 A54A AFAE") to "OpenPGP Public Key Primary Fingerprint" box and submit the change.

Update KEYS file under pinot-release-dist

Code Block

language	bash

$ cd pinot-release-dist
# Append the public key to KEYS file
gpg --list-sigs $GPG_KEY && gpg --armor --export $GPG_KEY >> KEYS

# Commit the change
svn commit -m "Update Pinot KEYS file for $GPG_KEY"

After following 1-3, you should have the following (after at most 24 hours delay. Certificates are synced from public domain once a day) If your key or fingerprint of the key does not show up correctly, then go back and verify the steps above. Did the key satisfy the SHA512 requirement? Can you try uploading it to other public servers?:

Your public key is listed in https://dist.apache.org/repos/dist/release/incubator/pinot/KEYS
Your public key is viewable at https://people.apache.org/keys/committer/<apache_ldap_id>.asc (e.g. https://people.apache.org/keys/committer/snlee.asc)
Your public key is viewable at https://people.apache.org/keys/group/pinot.asc

Clone Pinot code base

Clone the latest pinot code base in your local machine. Create a branch for your release from the point at which you wish to cut a release. his should be a commit hash that has been tested and verified in reasonable settings (e.g Pinot software has been tested to run in production with various use cases at a particular commit hash). For example:

Code Block

language	bash

$ git checkout -b release-<version> <commit-hash>
$ git push origin release-<version>

# Example
$ git checkout -b release-0.3.0-rc 9abf0a4b105d624ec310273c2bd7f52c1f3ce12a
$ git push origin release-0.3.0-rc

Run the test with sampling OFF, with the following changes (do not check in)

Code Block

language	bash

--- a/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/BaseClusterIntegrationTestSet.java
+++ b/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/BaseClusterIntegrationTestSet.java
@@ -297,10 +297,6 @@ public abstract class BaseClusterIntegrationTestSet extends BaseClusterIntegrati
     int maxNumQueriesToSkipInQueryFile = getMaxNumQueriesToSkipInQueryFile();
     try (BufferedReader reader = new BufferedReader(new FileReader(queryFile))) {
       while (true) {
-        int numQueriesSkipped = RANDOM.nextInt(maxNumQueriesToSkipInQueryFile);
-        for (int i = 0; i < numQueriesSkipped; i++) {
-          reader.readLine();
-        }
 
         String queryString = reader.readLine();
         // Reach end of file.
@@ -335,10 +331,6 @@ public abstract class BaseClusterIntegrationTestSet extends BaseClusterIntegrati
     try (BufferedReader reader = new BufferedReader(new FileReader(queryFile))) {
       while (true) {
         int numQueriesSkipped = RANDOM.nextInt(maxNumQueriesToSkipInQueryFile);
-        for (int i = 0; i < numQueriesSkipped; i++) {
-          reader.readLine();
-          queryId++;
-        }

Run the test, and make sure there are no failures (this can take 5-6 hours)

Code Block

language	bash

mvn clean test

Note
The update of LICENSE, NOTICE files, and the run of `mvn release:prepare` command, `mvn release:commit` command, etc. should be done on this branch, NOT on the master.

Prepare the release notes

Note
Please do not gather all the commit messages and turn that into release notes.

You need to prepare good release notes that highlights:

Important features added in the new release.
Any deprecated configuration or external methods, noting these these may be removed later.
Any configuration/methods that was deprecated earlier, and have been removed from this release.
Any special notes for upgrading from previous releases (e.g. for zero-downtime upgrade, please upgrade to release X before this one).
Other special notes

Here is a command you can use to get all the check-ins since the last release

Code Block

language	bash

git log 2a3860229^.. -- pinot-api -- pinot-azure-filesystem -- pinot-broker -- pinot-common -- pinot-connectors -- pinot-controller -- pinot-core -- pinot-hadoop -- pinot-hadoop-filesystem -- pinot-minion -- pinot-orc -- pinot-parquet -- pinot-server -- pinot-tools -- pinot-transport
# Assuming 2a3860229 was the commit hash in the master where the previous release was branched off
# Each argument following -- is a directory under pinot source tree (note that we have excluded the 'docs' directory

Once you have the commits, you are ready to come up with good material for release notes.

Draft the release notes in a text file (in markdown syntax) and keep it ready. You will need it in a later step.

Check out staging and release distributions

You need to check out Apache staging & release distribution directories for Pinot.

Code Block

language	bash

svn co https://dist.apache.org/repos/dist/release/incubator/pinot pinot-release-dist
svn co https://dist.apache.org/repos/dist/dev/incubator/pinot pinot-dev-dist

Info

Final PMC approved release will be placed in https://dist.apache.org/repos/dist/release/incubator/pinot. Do NOT upload anything here until voting from both dev@pinot and general@incubator pass.

Release candidates will be in https://dist.apache.org/repos/dist/dev/incubator/pinot. This link will be used for voting process.

Ensure access to Apache repository manager

You need to ensure that you have the access to Apache Repository Manager (Nexus). Try to log in with Apache user name and password.

Create PGP key and upload it to public servers

Note

It make take a few iterations (each time waiting for a day) to get this right, so we suggest that you get this started as early as possible in the release process, while you work on other steps (like updating LICENSE and NOTICE files).

If you have already done this for your apache user ID, you can skip this part.

You need to create a PGP key and upload it to public servers. Apache sites pick up the certificate from public repositories once a day. We strongly suggest that you read through this page first: https://www.apache.org/dev/release-signing.html. Here are the steps involved.

Update KEYS file under pinot-release-dist

Code Block

language	bash

$ cd pinot-release-dist
# Append the public key to KEYS file
gpg --list-sigs $GPG_KEY && gpg --armor --export $GPG_KEY >> KEYS

# Commit the change
svn commit -m "Update Pinot KEYS file for $GPG_KEY"

After following 1-3, you should have the following (after at most 24 hours delay. Certificates are synced from public domain once a day) If your key or fingerprint of the key does not show up correctly, then go back and verify the steps above. Did the key satisfy the SHA512 requirement? Can you try uploading it to other public servers?:

...

Updating LICENSE-binary file

...

Page tree

Versions Compared

Old Version 65

New Version 66

Key

Clone Pinot code base

Creating the key

Prepare the release notes

Check out staging and release distributions

Ensure access to Apache repository manager

Create PGP key and upload it to public servers

Creating the key

Update the key preference, if necessary

Uploading the generated key

Update the key preference, if necessary

Uploading the generated key

Clone Pinot code base

Prepare the release notes

Check out staging and release distributions

Ensure access to Apache repository manager

Create PGP key and upload it to public servers

Updating LICENSE-binary file

Page tree

Page History

Versions Compared

Old Version 65

New Version 66

Key

Clone Pinot code base

Creating the key

Prepare the release notes

Check out staging and release distributions

Ensure access to Apache repository manager

Create PGP key and upload it to public servers

Creating the key

Update the key preference, if necessary

Uploading the generated key

Update the key preference, if necessary

Uploading the generated key

Clone Pinot code base

Prepare the release notes

Check out staging and release distributions

Ensure access to Apache repository manager

Create PGP key and upload it to public servers

Updating LICENSE-binary file