This is the documentation on installing the Bigtop compile Toolchain on current git trunk.

For documentation of the Bigtop-1.0 Version see How to build Bigtop-1.0

For documentation of the Bigtop-1.1 Version see How to build Bigtop-1.1

Clone Bigtop repo

You can do it either from

Not Recommended: Setting up development environment from scratch:

Not recommended any more, suitable only if you plan to compile from a plain linux distribution

First, install a suitable puppet version (relative to bigtop root)

   # ./bigtop_toolchain/bin/puppetize.sh

next is installing the toolchain by a puppet call.

   # puppet apply --modulepath=`pwd`:/etc/puppet/modules -e "include bigtop_toolchain::installer"

(You may need to add "--parser future" for puppet 3.x or adjust the modulepath for the correct location of standard puppet modules)

Alternatively, if you already have sudo and a java runtime installed, you can bootstrap the environment using the provided Gradle wrapper script 

   # ./gradlew toolchain

Optional: Setting up bigtop/puppet docker images:

Only necessary if you want to rebuild the bigtop/puppet docker images, used as a base for bigtop/slaves. 

     # cd docker/bigtop-puppet/$BUILD_ENVIRONMENTS; ./build.sh

This is done by the CI Job:

https://ci.bigtop.apache.org/view/Docker/job/Docker-Puppet/

Optional: Setting up bigtop/slaves docker images:

Only necessary if you want to rebuild the bigtop/slaves docker images, used by the compile processes

     # ./gradlew -POS=$BUILD_ENVIRONMENTS bigtop-slaves

This is done by the CI Job:

https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/

Recommended: Using pre-built Docker images 

For current trunk (git master) the tags for bigtop/slaves are available: 

  • trunk-centos-6
  • trunk-centos-7
  • trunk-fedora-25
  • trunk-opensuse-42.1
  • trunk-ubuntu-16.04 
  • trunk-debian-8

The full image list is on Bigtop Docker Hub

The Dockerfiles to regenerate them are located in ./docker/bigtop-slaves.

Building bigtop-jsvc:

# ./gradlew bigtop-jsvc-pkg-ind

Building spark:

# ./gradlew spark-pkg-ind

You can find all valid gradle target using:

# ./gradlew tasks

For more details, refer to Quickstart Guide: Bigtop Integration Test Framework 2.0








  • No labels