Child pages
  • Build Creator
Skip to end of metadata
Go to start of metadata

Overview

Build Creator

The BuildCreator tool was devised to enable the easy generation of binary packages that include more than one source. The purpose of the build creator is to simplify the building of projects that require the combining of multiple sources. To aid discussion of the tool we shall look at the configuration file combining the Apache Qpid release with that of the BerkeleyDB Store plugin form JBoss. However, the tool is a general binary build tool written in Python.

Command Line arguments

There are various command line arguments, mainly to change the amount of logging the tool generates.

subprocess is required for this tool and is not present in versions prior to 2.4.0
usage: buildCreator.py [options]

options:
  -h, --help            show this help message and exit
  -c, --config=CONFIG
                        set configuration file : default = build.config
  -v, --verbose         enable verbose output
  -d, --debug           enable debug output
  -q, --quiet           Enable quiet ouptut
  -i, --ignore-errors   Ignore errors

There are a number of available targets:

Available Targets:
     distclean [source] - Remove all or specified retrieved source
     clean [source]     - Remove all or specified source build directory
     retrieve [source]  - Retrieve all or specified source
     prepare [source]   - Prepare all or specified source : i.e. extract archives
     patch [source]     - Patch all or specified source
     showbuilds         - List all builds
     build [build]      - Perform the build scripts for all or specified build
     release [build]    - Perform the release scripts for all or specified source
     full               - Perform clean, retrieve, prepare, patch, build, release for all builds (DEFAULT)

Targets

The default target is full which as listed above will perform all the required steps to create all the releases for the builds listed in the configuration. Most of the targets do exactly what they say above however a few have additional points of interest:

retrieve

The retrieve target pulls together all the source and patches and places this pristine copy in the builder/src directory. This is done so that the build can be repeatable performed with the untouched source.

prepare

The prepare target creates the builder/build tree and copies the source and patches for the later stages. If the source was not retrieved svn then the files are inspected and if an archive format is found this is expanded into the build directory.

release

The release scripts that are specified in the build files can contain a version keyword substitution $writeVersions(file). This will append details about the sources and patches used to generate this release artefact. If svn targets are used then the revision information will be added to the file.

Configuration Details

Two files are currently required by the tool the main configuration file and the build file.

Config file

The main configuration file (build.config by default) specifies the various sources, patches and builds that should be utilised.

The configuration file is used to define any common environment variables that may be used for substitution in the build scripts. The <sources> section contains a list of all source locations that the build will utilise. The <patch> section allows a number of patches to be specified to apply to the pristinely downloaded source to be modified.

NOTE: Limitiations
The build section can only contain <include> values not <build> elements directly

<builder>
    <environment>
        <[variable]>[value]</[variable]>
    </environment>

    <sources>
        <source>
            <name>[source-name]</name>
            <type>[source-type:svn|file|http|ftp]</type>
            <url>[value]</url>
            <path>[root offset, useful if to point the root of the source in to an archive output]</path>
        </source>
    </sources>
    <patches>
        <patch>
            <name>[patch-name]</name>
            <type>[source-type:svn|file|http|ftp]</type>
            <url>[value]</url>
            <source>[source name this patches]</source>
            <prefix>[patch prefix -p value]</prefix>
            <path>[root offset, useful if the base of the patch is not the root of the source]</path>
        </patch>

    </patches>

    <builds>
        <include>[string which is sent to ls to retrieve build include file so 'builds/*.build' works]</include>
    </builds>

</builder>

Source Section

The <source> section contains a number of required values.

Entry

Description

<name>

The name you wish to give to this build. This value should not contain spaces as it can be used in the build and release scripts as a variable to refer to the build location.

<type>

There are four types of source svn, file, http, ftp. These are explained further below

<url>

This is the URL or file path to this source

<path>

Optional path if you wish to move the root of the source. i.e. in to an archive file

Source Types

The <type> of the determines how the creator will post process the file.

Type

Description

svn

Uses svn to retrieve the given <url>, An optional <revision> element can be added to source to be used by the svn checkout

file

Uses cp to take a copy of the specified file.

http

Uses wget to retrieve the specified file or directory, it currently does not recurse.

ftp

Uses wget to retrieve the specified file or directory, it currently does not recurse.

Patch Section

The patch section contains all of the same entries as the Source Section. It also has two additional entries that are used in applying the patches.

Entry

Description

<source>

Source name that this patch entry should be applied to.

<prefix>

This is the patch prefix (-p) value used on the command line.

The <url> value here can either be a file or a directory. In the case that it is a directory all files contained in that directory will be treated as patch files and applied to the specified <source> with the given options.

Builds section

The final section in the config file is the <builds> section. Currently this can only contain a single <include> value. This specifies the build scripts definitions that should be included in this configuration. Any value that is a valid argument to ls can be used here. This allows multiple files to be included. Future work includes the direct embedding of the build configuration.

Build file

The build file contains scripts to perform the build and release of a desired release. There is no restriction on what can be performed during either section however only one <build> may be contained in each file.

<builds>
    <build>
        <name>[build-name]</name>

        <dependency>
            <source>[source-name]</source>
        </dependency>

        <targets>
            <build>
                <script><![CDATA[
<shell script>
]]>
                </script>
            </build>

            <release>
                <script><![CDATA[
<shell script>
]]>
                </script>
            </release>
        </targets>

    </build>
</builds>

Build Section

The build sction contains three elements all must be present to be valid.

Element

Desctiption

<name>

The name of this build. This is used to name directories on disk and to refer to this build directly on the command line.

<dependency>

This is a list of <source> entries that this build requires.

<targets>

This contains the two scripts one for <build> and one for <release>. The text in these scripts are executed in a shell rooted in the current directory.

Writing release scripts

To easy the writing of release scripts there are a number of variables that have been predefined for your use. The source <name> values can be used as variables to refer to the root of that source's build location, in the examples below that would mean that $qpid would be converted in to builder/build/qpid. As mentioned above you can also define variables in the <environment> section of the configuration. Release scripts can contain a version keyword substitution $writeVersions(file). This will append details about the sources and patches used to generate this release artefact. If svn targets are used then the revision information will be added to the file.

Example Files

<builder>
    <environment>
        <version>M3.0-beta</version>
    </environment>

    <sources>
       <source>
            <name>qpid</name>
            <type>file</type>
            <url>http://people.apache.org/~aidan/qpid/M3-beta/qpid-incubating-M3-beta.tar.gz</url>
            <path>qpid-incubating-M3</path>
        </source>
        <source>
            <name>bdb</name>
            <type>svn</type>
            <url>https://svn.jboss.org/repos/rhmessaging/store/branches/java/broker-queue-refactor/java/bdbstore</url>
        </source>
    </sources>

    <patches>
        <patch>
            <name>BDB-Classpath</name>
            <type>file</type>
            <url>/local/patches/bdb-qpid-run-classpath.diff</url>
            <source>qpid</source>
            <prefix>2</prefix>
            <path>qpid-incubating-M3/qpid/java/<path>
        </patch>

    </patches>

    <builds>
        <include>builds/*.config</include>
    </builds>

</builder>
<builds>
    <build>
        <name>qpid-broker</name>

        <dependency>
            <source>[source-name]</source>
            <source>bdb</source>
        </dependency>

        <targets>
            <build>
                <script><![CDATA[

pushd $qpid/java
ant -Dproject.version=$version build
popd

cp $qpid/java/build/lib/qpid-broker-$version.jar $bdb/lib
cp $qpid/java/build/lib/qpid-broker-test-$version.jar $bdb/lib
cp $qpid/java/build/lib/qpid-common-$version.jar $bdb/lib
cp $qpid/java/build/lib/qpid-systests-$version.jar $bdb/lib
cp $qpid/java/build/lib/qpid-perftests-$version.jar $bdb/lib
cp $qpid/java/build/lib/qpid-junit-toolkit-$version.jar $bdb/lib

cd $bdb
ant build

]]>
                </script>
            </build>

            <release>
                <script><![CDATA[
# Create build package
mkdir -p $release/$build-$version
cp -r $qpid/java/build/* $release/$build-$version
cp $bdb/build/qpid-bdbstore.jar $bdb/lib/je-3.3.62.jar $release/$build-$version/lib

# Build release artifact
cd $release/$build-version

# Create release revisions
echo "Qpid Broker Release : $version" > REVISIONS.txt
echo -n "Built:" >> REVISIONS.txt
date +%Y-%m-%d-%H%M >> REVISIONS.txt
$writeVersions(REVISIONS.txt)

cd ..

tar cvzf $build-$version.tgz $build-$version
]]>
                </script>
            </release>
        </targets>

    </build>
</builds>

  • No labels