You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 38 Next »

A work in progress...

Introduction

This document is intended for first-time reviewers and those who could benefit from additional information about the Apache software release process or specific tools needed to accomplish the review. See also How to Review a Release and Vote.

General getting-started information

Requirements:

Additional (optional) tools:

Verify your environment

Verify you have the correct versions of the following software. The versions must match the requirements in the module download instructions.

  • Maven

  • Java

Check the versions using the command line interface:

    mvn -- version
    java – version

Download the release candidate

  • Create a temporary directory for the release candidate.
  • Open the VOTE email, and click the release candidate link. This will open a web page with four additional links:  .zip (the release candidate), .asc (the signature file), .md5 (the md5 hash file), and .sha1 (the sha1 hash file).
  • Click on each of the four links and download the files to your computer. (You may need to save a file from the browser to text. Make sure the browser does not append ".txt" to the file.)

Alternatively, you can use wget to download the release candidate. (See Apache Stratos 3.0.0 Test Procedure.) 

BEFORE you unzip the release candidate

1a Check that Checksums are valid

There are three sources of checksums: (1) the VOTE email, (2) the release candidate .zip file, and (3) the checksum files (.md5 and .sha1) downloaded at the same time as the release candidate.

You can verify the checksums using a command line interface or a utility. Both methods are described below.

Command line interface

To generate checksums, navigate to the directory containing the .zip file and run the CertUtil command. The default gives you the SHA1 checksum. Append SHA512 and MD5, as shown below, to generate those checksums. Is the intent to make sure that the checksums from all 3 sources are identical?

CertUtil -hashfile 'apache-taverna-parent-2-incubating-source-release.zip'
CertUtil -hashfile 'apache-taverna-parent-2-incubating-source-release.zip' SHA512
CertUtil -hashfile 'apache-taverna-parent-2-incubating-source-release.zip' MD5

What's the best way to compare the checksums from the email and the .md5 and .sha1 files?

 MD5 and SHA Checksum Utility

When you double-click the .exe, a window opens (see below). Browse to the release candidate .zip file, and the checksums will be displayed. (Use the checkboxes to choose which checksums you want to see.) You can copy individual checksums, or use the Copy All button. At the bottom of the window, you can paste a checksum (e.g., either from the VOTE email or from the downloaded .md5 or .sha1 files) and the utility will verify the hash is the same as that from the .zip file.

MD5 and SHA Checksum Utility


1b Check that the PGP signature is valid

Each software artifact is signed using a PGP (Pretty Good Privacy) key. It is important to verify the key in copy of software you downloaded matches the key originally used to sign the software artifacdt.

You can verify the signatures using a command line interface or other software program (Kleopatra). Both methods are described below

Command line interface

  1. Download the Taverna key file from https://dist.apache.org/repos/dist/release/incubator/taverna/KEYS. (On Windows this can be done by typing Ctrl-S in the browser window and saving the file as keys.txt.)

  2. Import the key file into GPG:
    gpg --import keys.txt

     

  3. Download the .asc file

  4. Verify the .asc file matches the zipped release candidate. In general:

 

     gpg --verify file1.asc file2.zip

          And for example:

     gpg --verify 'apache-taverna- ... -source-release.zip.asc' 'apache-taverna- ... -source-release.zip'

 

 

AFTER you unzip the release candidate, but BEFORE you build

2 Check the commit ID matches the VOTE email

The idea here is to check that the commit ID in the downloaded file matches that in the VOTE email. You will use a command line terminal for at least some of these steps.

Approach 1 (clone the git repository, checkout the commit id, and compare to release candidate)

This approach uses Git commands to compare the downloaded release candidate to a cloned git repository. You will clone one copy and unzip the other copy in such a way that will "trick" Git into thinking you are comparing two versions that you have edited. (I'm missing part of the picture here. What is the end configuration you want before you use git status to compare files?)

  1. Make a new directory and change to that directory (e.g., mkdir 1 ; cd 1)
  2. Git clone that-repository (which repository?? from where??)
  3. Checkout the commit id from the repository you just created: git checkout <commit_id>
  4.  rm -rf* (remove a directory? I don't understand this step.)
  5. Unzip the release candidate (e.g., apache-taverna-parent-2-incubating-source-release.zip) (into the same directory?)
  6. mv apache-taverna-"/*. (one level up) (Move the release candidate up one level? Because it zips into a new folder?) 
  7. git status

Git will then check the checksums of every file and let you know what has "changed." 

If there are NO CHANGES, then the commit ID in the VOTE email matches the release candidate.

Approach 2 (download git commit from GitHub and compare to release candidate)

This approach uses the commit id from the VOTE email to download that commit from GitHub, which is then compared to the release candidate using the diff command.

  1. Search for the git commit corresponding to the email by browsing for it on GitHub. Use a URL that follows this pattern: https://github.com/apache/incubator-taverna-language/tree/<hash>, where <hash> is the commit has you want to download. 
  2. Click "Download Zip" and save the file to your computer.
  3. Make a new directory, change to the new directory, and unzip the GitHub file to the new folder. (e.g., mkdir 1 ; cd 1 ; unzip <filename>.zip)
  4. Move up a directory (cd .. ) and make a second new directory (e.g., mkdir 2)
  5. Change to the second new directory (e.g., cd 2) and unzip the release candidate (e.g., unzip release-candidate.zip)
  6. Move up one directory. When you do a directory listing (ls) you should see both of your new directories listed.
  7. Compare all files in the two new directories using the diff command:
    1. Linix: diff -uR 1 2
    2. Windows, GitBash: diff -r 1 2 (Windows CMD command line try FC)

The files that differ will be shown. If you do this after you build, make sure you don't have any target folders before diff-ing. (Run mvn clean to be sure.)

Again, if there are NO DIFFERENCES, then the commit ID in the VOTE email matches the release candidate.

3a Check the incubator disclaimer (INCUBATOR PROJECTS ONLY)

All Podlings must include a disclaimer indicating they are in incubation. To check a Taverna release candidate, open the DISCLAIMER file (in the top level release candidate folder) and verify the text matches the following: 

Apache Taverna is an effort undergoing incubation at the Apache Software 
Foundation (ASF), sponsored by the Apache Incubator PMC.
Incubation is required of all newly accepted projects until a further review 
indicates that the infrastructure, communications, and decision making process 
have stabilized in a manner consistent with other successful ASF projects.
While incubation status is not necessarily a reflection of the completeness 
or stability of the code, it does indicate that the project has yet to be 
fully endorsed by the ASF.

You can use a text editor (e.g., Notepad++) or an integrated development environment (IDE), such as Eclipse, to open the DISCLAIMER files. 

3b Check the file names include "incubating"

Check the unzipped release candidate to ensure the top-level distribution folder contains the word "incubating" (Is this all that can be checked before building? See 11 below.)

4 Check the top-level LICENSE and NOTICE files

LICENSE file

Check the text of the LICENSE file in the top-level release candidate folder, and ensure the Apache License matches the text of the required Apache License.

If additional, third-party licenses are listed below the required Apache License, ensure the licenses are allowed in Apache Software Foundation projects. (See also item 9, Check the dependencies - Review binary licenses.)

NOTICE file

Check the text of the NOTICE file in the top-level release candidate folder, and ensure it matches the following:

Apache Taverna Language
Copyright 2014-2016 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
Portions of this software were originally based on the following:
- Copyright 2010-2014 University of Manchester, UK
These have been licensed to the Apache Software Foundation under a software grant.

5 Check the source file headers

How does a reviewer know if a source file has been submitted directly to the ASF?

Code developed at the ASF

The ASF Source Headers web page details how to handle source file headers for code developed at the ASF. In short, the copyright must be removed (see requirements) and files should have the ASF license header text added to the header.

Specifically, check that each file has the ASF license header text and no copyright.  

Third-party files

Files used by the project, but not submitted by their owners directly to the ASF, are treated differently. Copyright notices stay in the header, and associated licenses must be part of the distribution.

Specifically, check that each file has a license (question) and that the license is allowed.

6 Check the provenance

TO DO What exactly is this, how does it differ from licensing, and how do you accomplish it?

7 Check that the source code does not include any binaries

Before building, the downloaded zip folder should contain no unexpected binary artifacts. For example, there should be no *.jar files. 

There may be some "expected" binary files, such as pictures and test workflows. These should be declared in NOTICE/LICENSE if they came from third-parties, e.g., if use use a Creative Commons-licensed JPEG.

Build

8 Build the release candidate(s) from source

Run mvn clean install

After successfully completing the verification steps above, use a command line interface and navigate to the folder containing the unzipped release candidate files. Execute the following command, which will build the release candidate and save the output to two text files: 

run mvn clean install > console.txt 2> err.txt

(The text file names can be anything.) This command will run several Maven goals, including automated tests. Do not skip the automated build tests. 

Review build completion status

BUILD FAILURE: At the completion of the mvn clean install process, if the message is "BUILD FAILURE," review both text files. Report the failure, along with any warning and error messages, on the release candidate's DISCUSS thread.

BUILD SUCCESS: If the completion message is "BUILD SUCCESS," review both text files. Report the success, along with any warning messages, on the release candidate's DISCUSS thread.

AFTER the (successful) build

9 Check the dependencies

Checking dependencies: 

  • Make sure dependencies are used and declared properly. (Use mvn dependency:analyze and look for warnings or errors.) 
  • Determine if there are any mismatches between resolved dependencies and the dependencyManagement section. (Use mvn dependency:analyze-dep-mgt and look for warnings or errors.)
  • Generate a list of licenses for each third-party dependency and check the licenses to make sure they meet Apache licensing requirements. (See Review Binary License below.)
  • You can create a list of dependencies by module using mvn dependency:list and generate a dependency tree to troubleshoot conflicts using mvn dependency:tree.

Review Binary Licenses. Create a list of third-party dependencies using the license:aggregate-add-third-party plugin and review the dependency licenses.)

  1. In a command line interface, change to the top level directory of the distribution (e.g., apache-taverna-language-<version>-incubator).
  2. Run the following Maven command: mvn license:aggregate-add-third-party. (On Windows, to save the output to a file, add > filename.txt to the end of the command.)
  3. This command will create THIRD-PARTY.txt files in each target folder (in the generated-sources/license subfolder).
  4. Review the THIRD-PARTY.txt files for unknown or disallowed licenses. Note: some unknown licenses have been determined to be allowed.)
    1. One way to do this: cat target/generated-sources/license/THIRD-PARTY.txt | sort

10 Verify the build produces the binaries

Quick check: browse the target folders and make sure there are not any extra folders. (For example, if we are voting on taverna-language there should not be any taverna-engine folders.)

Deeper check: ensure your target folders contains all the same *.jar files as those in the git repo? in the Maven repository? (Example link?)

At least one person should check that all staged JARs are the same as those built from the downloaded release candidate. (One approach is to do a recursive wget of the repository , and then compare the result of "find . -name '*jar'" in the wget-tree with */*/target/*.jar. See StackOverflow response.)

NOTE: Binary releases are considered "convenience only" and are not crucial for the vote: The source release is what everything else should be made from. However, in practical terms most people download the binaries from the Maven repository, so it is important this is checked at least once.

11 Verify all the *.jar files include the word "incubating"

Visually inspect all the *.jar files include the word "incubating" by opening all the /target folders. Is there an easier way to do this?

Are other files supposed to be labeled as well, or is it only the *.jar files?

Tips

 

  • How to check LICENSE, NOTICE, DISCLAIMER, and similar files?

    • Command line: Navigate to directory containing the file and use cat command to print contents to console (Example: cat LICENSE)
    • Text Editor/IDE: Use a text editor (e.g., Notepad++) or an integrated development environment (IDE), such as Eclipse, to open the LICENSE, NOTICE, DISCLAIMER, and similar files. 

  • How to capture the output of a command line interface command?

    • In a command line interface, send the console output and error messages to a text file:
      • GitBash example: mvn clean install > Console.txt 2> Err.txt

  • tbd


  • No labels