This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Page tree
Skip to end of metadata
Go to start of metadata

Getting Started

Clone the BigTop source code

$ git clone https://github.com/apache/bigtop.git

To deploy a cluster with the latest Bigtop 1.2.0 (docker based):

  • git checkout origin/branch-1.2
  • ./gradlew -Pconfig=config_ubuntu_xenial.yaml -Pnum_instances=1 docker-provisioner (to deploy to Ubuntu 16.04)
  • ./gradlew -Pconfig=config_debian8.yaml -Pnum_instances=1 docker-provisioner (to deploy to Debian 8)
  • ./gradlew -Pconfig=config_centos6.yaml -Pnum_instances=1 docker-provisioner (to deploy to CentOS 6)
  • ./gradlew -Pconfig=config_centos7.yaml -Pnum_instances=1 docker-provisioner (to deploy to CentOS 7)
  • ./gradlew docker-provisioner-destroy (to remove the containers)

To deploy a cluster with the latest Bigtop 1.1.0 (docker based):

  • git checkout origin/branch-1.1
  • ./gradlew -Pconfig=vagrantconfig_ubuntu.yaml -Pnum_instances=1 docker-provisioner (to deploy to Ubuntu 14.04)
  • ./gradlew -Pconfig=vagrantconfig_debian.yaml -Pnum_instances=1 docker-provisioner (to deploy to Debian 8)
  • ./gradlew -Pconfig=vagrantconfig.yaml -Pnum_instances=1 docker-provisioner (to deploy to CentOS 6)
  • ./gradlew docker-provisioner-destroy (to remove the containers)

For Bigtop 1.0.0:

To create a VM based BigTop Hadoop cluster:

$ cd bigtop/bigtop-deploy/vm/vagrant-puppet-vm

$ vim vagrantconfig.yaml

# Update the repo to be:

CentOS 6: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64

Debian 8: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/debian/8/x86_64

And update the component you'd like to deploy, for example: 

components: [hadoop, yarn, ignite-hadoop] (effective values)

$ vagrant up

# If using CentOS 7, you might need to disable firewall on each of the machines

$ vagrant ssh <machine_name, e.g. bigtop1>

$ sudo systemctl disable firewalld

$ sudo systemctl stop firewalld

DONE!

To create a Docker based  BigTop Hadoop cluster:

$ cd bigtop/bigtop-deploy/vm/vagrant-puppet-docker

$ vim vagrantconfig.yaml  

# Update docker image file to point to one of:

bigtop/deploy:centos-6

bigtop/deploy:debian-8

# Update the repo to be:

CentOS 6: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64

Debian 8: http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/debian/8/x86_64

And update the component you'd like to deploy, for example: 

components: [hadoop, yarn, ignite-hadoop] (effective values)

./docker-hadoop.sh --create 3

DONE!

Destroy clusters:

destroy VM cluster:

$ vagrant destroy -f

destroy Docker cluster:

$ ./docker-hadoop.sh --destroy

Demo video

  • No labels