Smalltop - The DConE Benchmarking Framework
How it works
Smalltop is derived from Apache Bigtop, originally designed to test Hadoop.
This framework is used to run performance benchmarks on the Distributed Coordination Engine (DConE). It is capable of:
- Deploying all nodes with Puppet
- Downloading the desired DConE JAR file for testing
- Building and driving tests with Bigtop
- Creating test reports
Node deployment with Puppet
Puppet allows us to declare a node configuration state. This is ideal for DConE node deployment because it guarantees that the final configuration state of a node is reached.
Node deployment can be performed by using this command:
- pdsh -R exec -w ssh:email@example.com.[xxx-xxx] puppet apply Smalltop/bigtop-deploy/puppet/manifests/site.pp
"xxx-xxx" are the nodes you want to configure for benchmarking. It is also assumed that these nodes contain the proper installations of Java, Ant, Maven, Puppet and Subversion.
The site.pp and dcone.pp are responsible for DConE node deployment. The configurations perform the following:
- Cleans nodes by stopping all Java processes and removing previously created directories from deployment
- Creates the necessary directories used for node deployment
- Copies and updates application properties files to directories
- Performs induction on nodes
- Creates and validates membership of nodes
- Assigns uuidProposers to begin proposing
Writing tests for Smalltop
Tests can be written in Groovy and saved under test-artifacts/src/groovy/org.apache.bigtop.itest/funny. The test driver method must contain @Test. Other junit test functions can be used such as @BeforeClass and @AfteClass, which execute before and after the test is performed.
Building tests with Bigtop
Testing is divided into two parts, test-artifacts and test-execution. Test-artifacts contain the performance tests that are written in Groovy script. Test-execution provide the instructions of which artifacts to test. Smalltop will automatically build and run all the tests contained in the directories specified in test-execution that contain junit tests (@Test).
The following commands must be used to build the DConE tests:
- mvn -f bigtop-tests/test-execution/smokes/funny/pom.xml clean
- mvn -f bigtop-tests/test-artifacts/funny/pom.xml clean install
- mvn -f bigtop-tests/test-execution/smokes/funny/pom.xml verify -Denforcer.skip=true
In this case, we are kicking off the Funny test, which is the first test written for Smalltop. The TestFunny.groovy file found under test-artifacts/funny contains the driving method that kicks off the test. In the current Funny test, for each node we want to perform benchmarking on, we instantiate DconeThread objects that will consume DConE output data and create reports. This is just one example of a benchmark test.
There are a variety of other classes contained within test-artifacts such as DataCollector.groovy and ReportCreator.groovy. These classes are used for data reporting. For instance, DataCollector, when instantiated, will gather the elapsed time and heap usage while agreements are being performed. DataCollector will return a String, which can then be passed into ReportCreator.groovy to write the string to a data report file.