Why Testing of a Release Artifact Matters

For an apache project, a VOTE on a release candidate is a very important process.  By voting (particularly for PMC members and committers), you are saying to the world that "yes, I have download, verified and tested using the project's procedure for testing".  Your +1, 0 or -1 vote is an indication of the success of the steps listed.  The more time that each individual spends reviewing the artifacts, the higher confidence we can have in the release.

Test with source code release

Test the Build

Clean up your environment

rm -Rf /tmp/cloudstack-cloudmonkey

Download the source build

Download the artifacts:

mkdir /tmp/cloudstack-cloudmonkey; cd /tmp/cloudstack-cloudmonkey
wget --no-check-certificate https://dist.apache.org/repos/dist/release/cloudstack/KEYS
wget --no-check-certificate https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.1.0/apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2
wget --no-check-certificate https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.1.0/apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2.asc
wget --no-check-certificate https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.1.0/apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2.md5
wget --no-check-certificate https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.1.0/apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2.sha

Install gpg (if needed): sudo apt-get install gpg

Import keys:  

This can be done by importing the keys stored in the source distribution's KEYS file:  gpg --import KEYS

Alternatively, you could download the signing keys (ID's found in the KEYS file) individually via a keyserver.  Ex:  gpg --recv-keys CC56CEA8

Verify signatures and hash files:

gpg --verify apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2.asc

This command should return "Good Signature".

gpg --print-md MD5 apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2 | diff - apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2.md5
gpg --print-md SHA512 apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2 | diff - apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2.sha

Each of these commands should return no output. If there is any output from them, then there is a difference between the hash you generated locally and the hash that has been pulled from the server.

Extract the source code and verify the contents:

Get the commit hash from the VOTE email (ex: 4cd60f3d1683a3445c3248f48ae064fb573db2a1). The value will change between releases.

Create two new temporary directories:

mkdir /tmp/cloudstack-cloudmonkey/git
mkdir /tmp/cloudstack-cloudmonkey/tree

Pull down the git repo:

git clone https://git-wip-us.apache.org/repos/asf/cloudstack-cloudmonkey.git /tmp/cloudstack-cloudmonkey/git
cd /tmp/cloudstack-cloudmonkey/git
git archive --format=tar --prefix=/tmp/cloudstack-cloudmonkey/tree/ <commit-hash> | tar Pxf -

Unpack the release artifact:

cd /tmp/cloudstack-cloudmonkey
tar xvfj apache-cloudstack-cloudmonkey-5.1.0-src.tar.bz2

Compare the contents of the release artifact with the contents pulled from the repo:

diff -r /tmp/cloudstack-cloudmonkey/apache-cloudstack-cloudmonkey-5.1.0-src /tmp/cloudstack-cloudmonkey/tree

Nothing should be different.

Verify the Code's License Headers

There are no automated tools at the moment, but this should help:

$ cd /tmp/cloudstack-cloudmonkey/apache-cloudstack-cloudmonkey-5.1.0-src
$ head -18 *.py */*.py

Verify that each file has the ASL

Run Some Tests

This assumes that you have a working CloudStack management server running that you can test against. Create a cloudmonkey config file along these lines (change as appropriate to your CloudStack installation)

$ cd /tmp/cloudstack-cloudmonkey/ 
$ cat > test.cfg << EOF 
cache_file = /tmp/cloudstack-cloudmonkey/cache 
log_file = /tmp/cloudstack-cloudmonkey/log 
asyncblock = true 
paramcompletion = false 
history_file = /tmp/cloudstack-cloudmonkey//history 
color = true 
prompt = >  
display = default 
secretkey = 
apikey =  
username = admin 
password = password  
path = /client/api 
host = localhost 
protocol = http 
port = 8080 
timeout = 3600 
expires = 300 


Start cloudmonkey

$ cd /tmp/cloudstack-cloudmonkey/apache-cloudstack-cloudmonkey-5.1.0-src/cloudmonkey
$ python cloudmonkey.py -c ../../test.cfg

Make sure you see '5.1.0' in the welcome message.

Try some initial tests

> list zones  
> list virtualmachines

Also make sure you try some asynchronous apis such as 'deploy virtualmachine'.

Edit the configuration file and enter valid values for the api key and secret key. Redo the tests.


If we made it this far, the release has at least been reasonably "smoke tested".

  • No labels