Getting Started

Clone the BigTop source code

$ git clone https://github.com/apache/bigtop.git

To deploy a cluster with the latest Bigtop 1.2.0 (docker based):

  • git checkout origin/branch-1.2
  • ./gradlew -Pconfig=config_ubuntu_xenial.yaml -Pnum_instances=1 docker-provisioner (to deploy to Ubuntu 16.04)
  • ./gradlew -Pconfig=config_debian8.yaml -Pnum_instances=1 docker-provisioner (to deploy to Debian 8)
  • ./gradlew -Pconfig=config_centos6.yaml -Pnum_instances=1 docker-provisioner (to deploy to CentOS 6)
  • ./gradlew -Pconfig=config_centos7.yaml -Pnum_instances=1 docker-provisioner (to deploy to CentOS 7)
  • ./gradlew docker-provisioner-destroy (to remove the containers)

To deploy a cluster with the latest Bigtop 1.1.0 (docker based):

  • git checkout origin/branch-1.1
  • ./gradlew -Pconfig=vagrantconfig_ubuntu.yaml -Pnum_instances=1 docker-provisioner (to deploy to Ubuntu 14.04)
  • ./gradlew -Pconfig=vagrantconfig_debian.yaml -Pnum_instances=1 docker-provisioner (to deploy to Debian 8)
  • ./gradlew -Pconfig=vagrantconfig.yaml -Pnum_instances=1 docker-provisioner (to deploy to CentOS 6)
  • ./gradlew docker-provisioner-destroy (to remove the containers)

For Bigtop 1.0.0:

To create a VM based BigTop Hadoop cluster:

$ cd bigtop/bigtop-deploy/vm/vagrant-puppet-vm

$ vim vagrantconfig.yaml

# Update the repo to be:

CentOS 6: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64

Debian 8: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/debian/8/x86_64

And update the component you'd like to deploy, for example: 

components: [hadoop, yarn, ignite-hadoop] (effective values)

$ vagrant up

# If using CentOS 7, you might need to disable firewall on each of the machines

$ vagrant ssh <machine_name, e.g. bigtop1>

$ sudo systemctl disable firewalld

$ sudo systemctl stop firewalld

DONE!

To create a Docker based  BigTop Hadoop cluster:

$ cd bigtop/bigtop-deploy/vm/vagrant-puppet-docker

$ vim vagrantconfig.yaml  

# Update docker image file to point to one of:

bigtop/deploy:centos-6

bigtop/deploy:debian-8

# Update the repo to be:

CentOS 6: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64

Debian 8: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/debian/8/x86_64

And update the component you'd like to deploy, for example: 

components: [hadoop, yarn, ignite-hadoop] (effective values)

$ ./docker-hadoop.sh --create 3

DONE!

Destroy clusters:

destroy VM cluster:

$ vagrant destroy -f

destroy Docker cluster:

$ ./docker-hadoop.sh --destroy

Demo video

Enable Port Forwarding

  • No labels