You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

The ManifoldCF Release Process

ManifoldCF is a little harder to release than most software projects because it has a fair number of connectors which are conditionally compiled. Thus, you cannot do a full release until the required dependencies are installed on your release machine. Luckily, for packages that are proprietary, stubs have been created which allow you to skip the installation of proprietary libraries in order to create a release.

Releases heretofore have been done on Windows (both US locale and Japan locale), and thus we feel that the release process is fairly robust for that platform. Batch scripts have been created that ease the process mightily. On Linux there are also scripts which have been used for one release.

Branches and tags

For any release, there should be a release branch. In the past, this branch has been created by the appropriate "svn copy" command from trunk. For example:

svn copy https://svn.apache.org/repos/asf/manifoldcf/trunk https://svn.apache.org/repos/asf/manifoldcf/branches/release-0.6-branch

See below for a description of some automated scripts that do this job now.

A major release will have its own release branch labeled with the release number (e.g. "release-0.6-branch"). A minor release will continue in the appropriate major release branch, and have the required changes pulled up from trunk one at a time. Thus 0.6.1 would be released from "release-0.6-branch" also.

When the time comes to build a release candidate, the release engineer should make an appropriately-labeled copy of the release branch in the "tags" area. For instance:

svn copy https://svn.apache.org/repos/asf/manifoldcf/branches/release-0.6-branch https://svn.apache.org/repos/asf/manifoldcf/tags/release-0.6-RC0

When the release is finally completed, the appropriate tag is then renamed to be the release name, e.g.:

svn move https://svn.apache.org/repos/asf/manifoldcf/tags/release-0.6-RC0 https://svn.apache.org/repos/asf/manifoldcf/tags/release-0.6

Preparing release branch

After you create the release branch, you need to set its version and update CHANGES.txt to be consistent with it being a release version. Here is the basic checklist.

  1. Update the build.xml - change the version number to the the planned release version ("XXXX")
  2. Update all the maven pom.xml files - formerly, we used "mvn versions:set -DnewVersion=XXXX", but that now seems to be unreliable, so see below
  3. Edit CHANGES.txt to mark the release (convert "XXXX-dev" to "Release XXXX")
  4. Check in these changes to the release branch

Since this process has been error-prone and is no longer even remotely automatic (due to the unreliability of mvn versions:set), I've written scripts that do the work.  Specifically:

 

  • create_release_branch.py <release_branch_version> <new_trunk_version>
  • update_release_branch.py <release_branch_major_version> <new_branch_version>

 

The first script makes a copy of trunk and to "branches/release-<version>-branch".  It modifies build.xml, CHANGES.txt, and all the pom.xml files on both branch and trunk.  The second script operates on an existing release branch to mark a bug fix release.  It updates build.xml, CHANGES.txt, and all the pom.xml files on "branches/release-<major-version>-branch".  Both scripts can be found in svn under https://svn.apache.org/repos/asf/manifoldcf/release-scripts .

Before releasing

Before you cut a release, it's always a good idea to run all the tests you can on the software. Also necessary is a "rat" run, which detects unknown or incorrect licenses on source files. See the complete checklist below.

  1. Download the dependencies for building and testing (ant make-core-deps make-deps). You will need svn 1.7+ installed for this step.
  2. Check licenses (ant rat-sources). You are interested in verifying that there are zero files with unapproved licenses. (This comes at the very start of a long report.) You will need Apache Rat installed for this step.
  3. Run standard integration tests (ant test-dr test-hs). If you have PostgreSQL installed and MySQL, run those too (ant test-pg test-my).
  4. Run the UI tests (ant uitest). You'll need Python 2.7+ for these.

If everything looks good, you're ready to go ahead and cut the release!

Building ManifoldCF for release

NOTE: In order to proceed, you MUST have svn 1.7 or higher installed! The ant build scripts require this in order to be able to apply patches on Windows platforms. You will also need GnuPG installed (from http://www.gnupg.org/download/), and have your signing key created and installed (read this: http://www.apache.org/dev/release-signing). Also needed is the utility "pscp.exe", and a properly configured public/private key signon to people.apache.org (which is required to upload artifacts; read here how to set that up: https://www.debian-administration.org/article/530/SSH_with_authentication_key_instead_of_password).

The first step of the build process is to check out the release branch into a work area. Technically it is more correct to check out the tag, but in practice that's not necessary because only the release engineer should be working on the release branch. All other development should be taking place in trunk.

svn co https://svn.apache.org/repos/asf/manifoldcf/release-0.6-branch myworkarea

Then, check out the release scripts:

svn co https://svn.apache.org/repos/asf/manifoldcf/release-scripts

The release scripts perform all the required fetching of dependencies automatically, through ant. But they do not know how to deal with proxies, so if you have a proxy you will need to figure out how to deal with that.

Next, you will need to create a place to put the release in your people.apache.org public_html area. This directory should be named "apache-manifoldcf-<release_name>". The release candidate will be copied there.

The next step is to fire off the release script itself. You will need to supply your people.apache.org username also, as well as your SSH private key:

cd myworkarea
../release-scripts/release.bat 0.6 kwright c:\passwd\kwright-people.apache.org.ppk

After being prompted a few times for your key file password, and for your people.apache.org password, your release candidate should be created and copied to an appropriate subdirectory in your user area on people.apache.org, e.g. http://people.apache.org/~kwright/apache-manifoldcf-0.6.

Finally, create a tag corresponding to the release candidate, as previously described. Obviously release candidates should start at RC0 and count up sequentially.

Calling the vote

Once the artifact has been uploaded to people.apache.org, you should call a vote to release the artifact. This involves sending an email to dev@manifoldcf.apache.org, with the title "[VOTE] Release Apache ManifoldCF xxxx RCnnn". In the vote email, mention where the artifacts can be downloaded from (http://people.apache.org/~<username>/apache-manifoldcf-xxxx), as well as the full svn URL of the tag you created (https://svn.apache.org/repos/asf/manifoldcf/tags/release-xxxx-RCnnnn). The vote must have three +1's from committers, and must be open at least 72 hours.

If the vote does not pass

If the vote does not pass, this usually implies that fixes will be made and a new RC created. Fixes usually are committed to trunk first and then pulled up into the release branch using the "svn merge" command, e.g. "svn merge -c ZZZZ https://svn.apache.org/repos/asf/manifoldcf/trunk". Before the creating of the next RC, you should also log into people.apache.org again, and move the old artifacts from the public-html/apache-manifoldcf-XXX directory to public-html/apache-manifoldcf-XXX/RCnnnn. Then, upload the new RC as before.

If the vote passes

When the vote passes, you should check the artifacts from your public-html area directly into the svnpubsub svn. DO NOT REGENERATE THE ARTIFACTS - the community has voted on them and they are the only legal copies you should distribute. Once that is done, you should also use "svn move" to rename the artifact RC tag you created to remove the -RCnnnn suffix from it.

Updating trunk post-release

Trunk used to need to be updated to reflect the fact that it now represents the next release. There were four things you needed to do in order to complete this step:

  1. Update the build.xml - change the version number to the next planned release version ("XXXX-dev")
  2. Update all the maven pom.xml files - "mvn versions:set -DnewVersion=XXXX-SNAPSHOT"
  3. Edit CHANGES.txt to mark the release and start a new line marking the end of the next release ("XXXX-dev")
  4. Check in these changes to trunk

These tasks are now done automatically by the create_release_branch.py script, at the time the release branch is created.

 

  • No labels