The new Bigtop integration test framework aims to provide a one stop seamlessly integrated Hadoop ecosystem build and test solution to maximize the user experience . With Integration Test Framework 2.0, everything is under your control at a single entry of Gradle command.
Experience it by trying with following examples!
- Make sure you have docker suits up. Go to docker website(https://docs.docker.com/) to install docker on your environment.
Download apache bigtop latest release at http://bigtop.apache.org/download.html. Untar and get into the directory:
To build Hadoop ecosystem packages, just do:
If you'd like to create a repository(yum or apt) at local, just do:
Note that the build system does not know which system(yum or apt) you'd like to build. But you can hint it with -POS property.
Typically we'll just do it together along with the build:
Deploy and Test
To deploy a Bigtop stack inside docker containers using Bigtop Docker Provisioner, just do:
Typically what we want to do with provisioner is to deploy a cluster and then run smoke tests. The following example deploys Spark on Hadoop and runs Spark smoke test:
To be noticed that, anything not specified is default to what specified in the config file. If no config file specified, default to config_centos-7.yaml
Another typical case is the Build → Deploy → Test lifecycle. You can do it in one command like this:
Build from branch and commit hash
Another developer friendly feature added is to build packages from a git branch, or even a commit hash. To do so, refer to the document here: