Jena Release Process

This document outlines the procedure for producing a Jena release

Useful input:

The original Jena release under Apache was put together pragmatically to get the first release done, see email "Release Process Notes". This document gives a revised and improved version of these original notes and covers some gotchas that may catch people out

Creating a Release Candidate

You must use maven3, maven2 is unreliable, sometimes missing out the source-release

Pre-Release Preparation

Email the dev@ list to say as release is imminent, to let people finish up anythign they want to get into the next release.

Run RAT to check for missing licenses, you can download RAT binaries from the Apache Creadur project
Jena is not RAT-clean - there are many very small testing files that don't have a license statement in them. It is very important to look for any java files without a license - there should be none.

Check vesion numbers and dependencies between modules.

These should be the current SNAPSHOTs.

Step 1 - Checkout a clean copy of the codebase

svn co https://username@svn.apache.org/repos/asf/jena/trunk/ JENA
cd JENA

Don't forget the https - the release candidate creation process will need to do some svn commits so must use a read/write connection to SVN

Currently, jena-iri, jena-core, jena-arq, and jena-tdb are included in the download, apache-jena is the module that creates the distribution, jena-fuseki gets built as well but is a separate distribution binary.

The codetree includes all active modules, even those not included in the release.

Delete the jena-sdb, jena-larq trees from the local copy since they should not be included in the source-release zip that the release candidate creation process will create as we have not yet officially released them as Apache modules.

rm -rf jena-sdb jena-larq

Step 2 - Make a clean maven repo

It is strongly suggested that you create a brand new maven repository and create a settings file to go with it, this settings file needs to contain the following:

<?xml version="1.0" encoding="UTF-8"?>
<settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd" xmlns="http://maven.apache.org/SETTINGS/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <localRepository>${user.home}/.m2/jena-release</localRepository>
  <servers>
    <server>
        <id>apache.releases.https</id>
        <username>username</username>
        <password>password</password>
    </server>
  </servers>
</settings>

Subsequent steps will refer to this settings file as settings.xml

This will create a special jena-release repo under your standard maven install, it also configures the credentials you will need in later steps. You may need to ensure that the jena-release directory exists before carrying on, if you've previously done a release candidate for a prior release you should ensure that the contents of this repository are emptied before continuing.

Step 3 - Check/Set Versions

If still separate version numbers for each module:

The parent POM should be the SNAPSHOT.
The dependencies should be the SNAPSHOT.

Normally, this is a no-work stage.

Step 4 - Maven Release Dry run

Run the following maven command:

mvn -s settings.xml release:clean release:prepare -DdryRun=true -Papache-release

The defaults for "mvn release:prepare" should mostly be fine. The only one you should change is when prompted for the SCM tag enter Jena-Ver-RC_X_ where Ver is the release version you are preparing a release candidate for and X is the release candidate number.

Check when running this step but if the project is setup correctly no other changes should be needed.

Step 5 - Check the release setup.

Check the produced release.properties file, in particular check the versions numbers and the SCM tag all look OK.

Then check the artifacts produced in the various target directories:

JENA/target/apache-jena-VER-source-release.zip
JENA/apache-jena/target/apache-jena-VER.tar.gz and .zip
JENA/jena-fuseki/target/jena-fuseki-VER-distribution.tar.gz and .zip

Step 6 - Run Maven Release

Run the following maven commands:

mvn -s settings.xml release:clean release:prepare  -Papache-release
mvn -s settings.xml release:perform  -Papache-release

Common/Known Issues with Maven Release

Problem tagging SCM

On "mvn release:clean release:prepare -Papache-release"

If you see something like:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.1:prepare (default-cli) on project jena-tdb: Unable to tag SCM
[ERROR] Provider message:
[ERROR] The svn tag command failed.
[ERROR] Command output:
[ERROR] svn: No such revision 1245420

It's because the Apache SVN mirrors aren't quite sync'ed. Resume with:

mvn -s settings.xml release:prepare -Papache-release

and it will continue where it left off. Note : no "release:clean" here.

Problem transferring artifacts

If you receive errors about failing to transfer artifacts and the error mentions response 401 this means you are not properly authenticating to the Apache Maven repository. Ensure that you are using a correctly configured settings.xml file per Step 2

Recovering from a serious release failure

If all else fails:

mvn release:rollback

Or in the worse case reset the POM manually ; commit changes.

svn rm -m "Reset"  ... the SVN tagged version

Clean the maven local repository for stuff from the build

rm -rf .m2/repository/org/apache/jena/jena-MOD/VER/ etc
mvn clean

Step 7 - Close the Staging Repository

Finding the Repository

The staging repository can be found by going to http://repository.apache.org - you should be able to log in with your normal Apache login. Hit the Staging Repositories link in the sidebar to see the list of repositories, if the release went OK you should see a org.apache.jena-VER (u: username a: your IP) where VER is the version whose release candidate you are preparing.

Tick the box next to it in the list and it will display the artifacts in that repository in a separate pane below, browse through this to check that all the expected artifacts are present

Closing the Repository

Close it by selecting it in the Staging repositories list and then clicking the Close link. Note that if you try to click on the repository URL prior to closing it you will receive a Not Found error from the server, you cannot browse the repository by URL until you close it.

Once closed you can click on it, copy the URL from the address bar as you will need to reference this later.

Step 8 - Get the distribution tools

Before you can prepare the distribution you need to get the dist tools, check these out into a clean location:

svn co https://svn.apache.org/repos/asf/jena/dist/

KEYS File

If you have not made a release or added your key to the KEYS file previously you should do this now:

gpg --list-sigs "Your Name" >> KEYS
gpg --armor --export "Your Name" >> KEYS

Remember to commit the KEYS file after you have done this. See Signing Releases for more notes on the KEYS file.

Step 9 - Build the distribution hierarchy.

Firstly you need to fetch the staging repository in order to have the release candidate artifacts available:

mkdir -p repository.apache.org/content/repositories/orgapachejena-NNN/org/apache/jena
wget -e robots=off --wait 1 --mirror -np https://repository.apache.org/content/repositories/orgapachejena-NNN/org/apache/jena

Note: Remember to replace NNN appropriately with the actual ID number of the staging repository you closed back in Step 7

You need a version of wget with HTTPS support for the above to work, if you built and installed wget yourself you must have added the --with-ssl or --with-ssl=openssl option when you ran the configure script. You cannot retrieve the content from the repository using wget without HTTPS installed

After this is done move the mirrored artifacts like so:

mv repository.apache.org/content/repositories/orgapachejena-NNN/ REPO

You are now ready to run the actual script which requires two arguments, the first is the version of the main Jena distribution and the second is the version of the Fuseki distribution. If you want to dry run you can add echo as a third argument:

./dist.sh 2.7.3 0.2.4

The script will create a new directory dist/ under which you should find all the artifacts appropriately laid out.

Step 10 - Upload the Release Candidate

You now need to copy the proposed release candidate to your public web area on people.apache.org e.g. http://people.apache.org/~rvesse

Note that if accessing your URL returns a 404 rather than a directory listing you first need to ssh into people.apache.org and run the following:

mkdir /home/username/public_html

If this is done you can use your favourite SCP/SFTP tool to upload the release candidate, for example:

scp -r dist people.apache.org:public_html/Jena-VER/

Step 11 - Local Clean Up

Go and clean up your local maven repositories and be sure to do a svn update on your working copy of trunk to grab the updated POMs

Step 12 - Call the VOTE

Look at the email archives for sample vote emails e.g.

Make sure to include links to the Staging Repository, SCM Tag, Proposed Release Candidate URL and KEYS file URL

Step 13 - Rebuild the jena development build on Jenkins

This should happen fairly automatically, check http://builds.apache.org

Making a Release

If a Release Candidate vote passes successfully then you need to follow these steps to make it an official release, see Apache Releases FAQ if you need more detail on how to do any of these steps.

Step 1 - Release the staging repo

Go to repositories.apache.org and login with your Apache account, use the Staging Repositories link from the sidebar to access the list of staged repositories. Tick the box next to the Apache Jena repository that was created when you prepared the release candidate. Hit the Release link at the top of the page, enter "Release Jena X.Y.Z" in the description box and hit OK. You should get a Repositories successfully released message after a few seconds.

Step 2 - Install the release files (source and binaries).

Like everything else Apache distributions are made via a svnpubsub process, firstly check out the Jena distribution area to a clean directory:

svn co https://user@dist.apache.org/repos/dist/release/jena/ SOMEDIR

Once this is checked out you firstly need to remove the old version from the checked out copy, in the dist directory the following commands will do this for you:

cd source
svn delete jena*
cd ../binaries
svn delete apache*
svn delete jena*
cd ../

Then you can copy the proposed distribution area you made in Step 9 of the release candidate into the checked out copy:

cp -R ../PROPOSED-DIST .

Then do the following to ensure all the new artifacts are added to svn:

cd source
svn add jena*
cd ../binaries
svn add apache*
svn add fuseki*
cd ../
svn commit --message "Committing the Jena VER artifacts to the Jena release area"

Step 3 - Reset versions in JIRA

Go into JIRA and mark the versions as released, ensure the new snapshot versions are entered if they aren't already.

To do this you should log into JIRA and then go to the Administration link in the top right corner. Click on Jena under the Projects list on the Administration page, then on the Project config page click on Versions in the left hand sidebar. If the releases don't already have a release date listed for them you need to click on the little cog on the right hand side of the row containing the version being released and select "Release" from the drop down.

If versions have not been added already for the next releases you should add those now.

Step 4 - Clean up the snapshot repo

TODO - Ask Andy what this means?

Step 5 - Update the website

Update the Javadoc

For javadoc it is better to use the publish script as a lot of change can overwhelm web CMS. It is easier to delete the old java doc and add the new, note that this can take a while on the "svn commit"

You can checkout a copy of the website by doing the following:

svn co https://username@svn.apache.org/repos/asf/jena/site/trunk/ WEBSITE

Then in the website directory do the following to remove the old documentation:

cd content/documentation/javadoc/
cd arq
svn delete *
cd ../fuseki
svn delete *
cd ../jena
svn delete *
cd ../tdb
svn delete *
cd ../
svn commit --message "Deleting old javadoc"

You can then go ahead and copy the updated documentation from your release candidate area like so:

cp -R RC/jena-arq/target/apidocs arq
cd arq
svn add *
cd ../
cp -R RC/jena-fuseki/target/apidocs fuseki
cd fuseki
svn add *
cd ../
cp -R RC/jena-core/target/apidocs jena
cd jena
svn add *
cd ../
cp -R RC/jena-tdb/target/apidocs tdb
cd tdb
svn add *
cd ../
svn commit --message "Adding updated javadoc"

You then need to go the CMS to actually publish the updated docs, firstly check the status of the site build by going to http://ci.apache.org/builders/jena-site-staging. If you see anything listed under Current Builds then the build is still running, wait a few minutes and then refresh (the build can take a long time for big changes).

Once there are no Current Builds running click on the most recent build listed to ensure it matches your commit message and has succeeded. You can now go to https://cms.apache.org/jena/publish and enter a commit message stating that you are publishing the updated Javadoc. Note that you can view the diff if you want but it may take a long time for the CMS to generate it especially if a lot of documentation has changed.

Download Pages

Check the following pages to ensure they are up to date. You can use the web CMS to make the changes and should go ahead and publish immediately:

Step 6 - Release

Wait 24 hours to allow all the Apache mirrors to sync up.

Announce via email to:

usesr@jena.apache.org
announce@apache.org
semantic-web@w3.org (if significant changes)

Voting on a Release

When voting on a release candidate the following points need to be checked:

is the GPG signature fine?
is there a source archive?
can the source archive really be built?
is there a correct LICENSE and NOTICE file in each artifact (both source and binary artifacts)?
does the NOTICE file contain all necessary attributions?
check the dependencies.
do all the tests work?
if there is a tag in the SCM, does it contain reproduceable sources?

Child pages

Release Process