This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Page tree
Skip to end of metadata
Go to start of metadata

This is the documentation on installing the Bigtop compile Toolchain on current git trunk.

For documentation of the Bigtop-1.0 Version see How to build Bigtop-1.0

For documentation of the Bigtop-1.1 Version see How to build Bigtop-1.1

Clone Bigtop repo

You can do it either from

Not Recommended: Setting up development environment from scratch:

Not recommended any more, suitable only if you plan to compile from a plain linux distribution

First, install a suitable puppet version (relative to bigtop root)

   # ./bigtop_toolchain/bin/

next is installing the toolchain by a puppet call.

   # puppet apply --modulepath=`pwd`:/etc/puppet/modules -e "include bigtop_toolchain::installer"

(You may need to add "--parser future" for puppet 3.x or adjust the modulepath for the correct location of standard puppet modules)

Alternatively, if you already have sudo and a java runtime installed, you can bootstrap the environment using the provided Gradle wrapper script 

   # ./gradlew toolchain

Optional: Setting up bigtop/puppet docker images:

Only necessary if you want to rebuild the bigtop/puppet docker images, used as a base for bigtop/slaves. 

     # cd docker/bigtop-puppet/$BUILD_ENVIRONMENTS; ./

This is done by the CI Job:

Optional: Setting up bigtop/slaves docker images:

Only necessary if you want to rebuild the bigtop/slaves docker images, used by the compile processes

     # ./gradlew -POS=$BUILD_ENVIRONMENTS bigtop-slaves

This is done by the CI Job:

Recommended: Using pre-built Docker images 

For current trunk (git master) the tags for bigtop/slaves are available: 

  • trunk-centos-6
  • trunk-centos-7
  • trunk-fedora-25
  • trunk-opensuse-42.1
  • trunk-ubuntu-16.04 
  • trunk-debian-8

The full image list is on Bigtop Docker Hub

The Dockerfiles to regenerate them are located in ./docker/bigtop-slaves.

Building bigtop-jsvc:

# ./gradlew bigtop-jsvc-pkg-ind

Building spark:

# ./gradlew spark-pkg-ind

You can find all valid gradle target using:

# ./gradlew tasks

For more details, refer to Quickstart Guide: Bigtop Integration Test Framework 2.0

  • No labels