This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Page tree
Skip to end of metadata
Go to start of metadata

Build system and stack DSL

Apache Bigtop has moved away from the original make-based system at the end of 2014. We have decided in favor of more human-oriented and a way more powerful Gradle-based approach. Thanks to the amazing power of Apache Groovy, it was easy to create a descriptive DSL for software stacks under the Bigtop control. By default, the stack definition is located in at the top-level project's folder. Here's a snipped of the self-documented DSL 


bigtop { // *the name should be change: the parsing code depends on it*
  version = "STACK-VERSION" // *required*
  stack { // *required* Fundamental properties of the Stack: JDK, SDK, GDK, etc
    'jdk' { version = '1.8'; version_base = version }
    'scala' { version = '2.10.4'; version_base = version }
  apache { // *required* These shoudn't be modified unless ASF Infra demands changes
  git { // *optional* This is a global setting to access protected git repositories, 
        //            can be specified per component as well
    user = "john_doe"
    token = "john's access token"
  components { *required; preserve the name* if empty, nothing will be built
    'label' { // label *SHOULD* be the same as the name; otherwise some tasks will fail
      name    = 'component1' // *required* the name of the component
      // 'pkg' value is optional and will be set to that of 'name' i.e. [pkg := name]
      pkg     = name // *optional* and will be set to the 'name' value
      // 'base' is required; [pkg := base ]; [release := 1 ]
      version { base  = 'x.y.z'; pkg   = base; release = 1 }
      tarball {
        source      = "apache-component1-${version.base}.tar.gz"
        // It is advised to use different destination filenames to avoid
        // clashes when working with git repos and downloading the artifacts
        // from the branches with the same names.
        destination = source
      url { // *optional*
        download_path = "/component1/component1-${version.base}"
        site          = "${apache.APACHE_MIRROR}/${download_path}"
        archive       = "${apache.APACHE_ARCHIVE}/${download_path}"
      git {
        // Setting the info to access a git repository. Ref is any valid git reference.
        // If git repo information is provided, the *url* element above will be ignored.
        repo  = ""
        ref   = "branch-name"
        // *dir* defines the name of the top-level folder inside of the tar-ball archive.
        // if set to null, the directory name will be set to tar-ball.dist without
        // the .tar* suffix
        // This setting is important to allow build to locate unpacked source code
        dir   = "${name}-${version.base}-src"
        // *optional*
        // You can setup repo-specific user credentials overriding any global settings
        user  = "john_doe"
        token = "john's access token"

A special note needs to be made about building components with -SNAPSHOT versions. Because RPM format doesn't permit symbol - in the package name, one needs to remove the suffix from the package version string. Fortunatelly, it is very easy to do in a Groovy DSL:

version { base = '2.7.4-SNAPSHOT'; pkg = base-"-SNAPSHOT"; release = 1 }

Yes, you can substract -SNAPSHOT token from the base version string.

Setting up the CI environment

How to build...

Deploying Build Artifacts

  • No labels