Build system and stack DSL

Apache Bigtop has moved away from the original make-based system at the end of 2014. We have decided in favor of more human-oriented and a way more powerful Gradle-based approach. Thanks to the amazing power of Apache Groovy, it was easy to create a descriptive DSL for software stacks under the Bigtop control. By default, the stack definition is located in bigtop.bom at the top-level project's folder. Here's a snipped of the self-documented DSL 

 

bigtop { // *the name should be change: the parsing code depends on it*
  version = "STACK-VERSION" // *required*
  stack { // *required* Fundamental properties of the Stack: JDK, SDK, GDK, etc
    'jdk' { version = '1.8'; version_base = version }
    'scala' { version = '2.10.4'; version_base = version }
  }
  apache { // *required* These shoudn't be modified unless ASF Infra demands changes
    APACHE_MIRROR = "http://apache.osuosl.org"
    APACHE_ARCHIVE = "https://archive.apache.org/dist"
  }
  git { // *optional* This is a global setting to access protected git repositories, 
        //            can be specified per component as well
    user = "john_doe"
    token = "john's access token"
  }
  components { *required; preserve the name* if empty, nothing will be built
    'label' { // label *SHOULD* be the same as the name; otherwise some tasks will fail
      name    = 'component1' // *required* the name of the component
      // 'pkg' value is optional and will be set to that of 'name' i.e. [pkg := name]
      pkg     = name // *optional* and will be set to the 'name' value
      // 'base' is required; [pkg := base ]; [release := 1 ]
      version { base  = 'x.y.z'; pkg   = base; release = 1 }
      tarball {
        source      = "apache-component1-${version.base}.tar.gz"
        // It is advised to use different destination filenames to avoid
        // clashes when working with git repos and downloading the artifacts
        // from the branches with the same names.
        destination = source
      }
      url { // *optional*
        download_path = "/component1/component1-${version.base}"
        site          = "${apache.APACHE_MIRROR}/${download_path}"
        archive       = "${apache.APACHE_ARCHIVE}/${download_path}"
      }
      git {
        // Setting the info to access a git repository. Ref is any valid git reference.
        // If git repo information is provided, the *url* element above will be ignored.
        repo  = "https://github.com/apache/bigtop.git"
        ref   = "branch-name"
        // *dir* defines the name of the top-level folder inside of the tar-ball archive.
        // if set to null, the directory name will be set to tar-ball.dist without
        // the .tar* suffix
        // This setting is important to allow build to locate unpacked source code
        dir   = "${name}-${version.base}-src"
        // *optional*
        // You can setup repo-specific user credentials overriding any global settings
        user  = "john_doe"
        token = "john's access token"
      }
    }
  }
}

A special note needs to be made about building components with -SNAPSHOT versions. Because RPM format doesn't permit symbol - in the package name, one needs to remove the suffix from the package version string. Fortunatelly, it is very easy to do in a Groovy DSL:

version { base = '2.7.4-SNAPSHOT'; pkg = base-"-SNAPSHOT"; release = 1 }

Yes, you can substract -SNAPSHOT token from the base version string.

Setting up the CI environment

How to build...

Deploying Build Artifacts

  • No labels