Skip to end of metadata
Go to start of metadata

TBD: include pointers to "good" unit tests in the project

Geode is tested with a combination of JUnit, DUnit (distributed unit) and regression tests. JUnit tests are written as with any other project while DUnit tests use a multi-JVM environment (see Distributed Unit Tests).

JUnit tests are divided into two main categories: UnitTest and IntegrationTest.

1. UnitTest: is a test with very narrow and well defined scope. Any complex dependencies and interactions are stubbed or mocked.

  • Should use JUnit 4 syntax
  • File name ends with *Test
  • Should use Category annotation of type UnitTest
  • Should complete in milliseconds 
  • Should generally test a single class
  • Typically uses white-box testing, uses Mocks/Fakes and helps guarantee internal quality (quality of code and class design)
  • Follows "A Set of Unit Testing Rules" by Michael Feathers
    • A UnitTest should not do any of the following:
      • communicate with a database
      • communicate across the network
      • access the file system
      • prevent the running of other unit tests in parallel
      • require anything special in the environment (such as editing config files or running an external process)

2. IntegrationTest: a test involving inter-operation of components or subsystems.

  • Should use JUnit 4 syntax
  • File name ends with *IntegrationTest
  • Should use Category annotation of type IntegrationTest
  • May take longer than a UnitTest but should generally complete in seconds
  • May test a single class or any combination of classes including end-to-end testing of Geode
  • Typically uses black-box testing but may include some white-box testing and helps guarantee external quality (feature delivers value to User)
  • An IntegrationTest may do any of the following:
    • communicate across the network
    • access the file system
    • require anything special in the environment (such as editing config files or running an external process)
  • Good examples in Geode:
    • RegionExpirationIntegrationTest

DUnit tests are a special type of JUnit test involving multiple JVMs. This supports writing tests for a Geode cluster.

3. DistributedTest: a test involving multiple members of a distributed system.

  • Should use JUnit 4 syntax
  • File name ends with *DistributedTest or *DUnitTest
  • Should use Category annotation DistributedTest
  • Should generally complete in seconds
  • May test interactions between multiple JVMs in a Geode cluster or via networking or via file system
  • Typically uses black-box testing but may include some white-box testing and helps guarantee external quality (feature delivers value to User)
  • A DistributedTest may do any of the following:
    • communicate across the network
    • access the file system
    • require anything special in the environment (such as editing config files or running an external process)

Regression tests are typically an IntegrationTest or DistributedTest written to reproduce a specific bug and verify the fix of that bug.

4. RegressionTest: an IntegrationTest or DistributedTest that reproduces a specific bug and verifies its fix.

  • Should use JUnit 4 syntax
  • File name ends with *RegressionTest
  • Should use Category annotation of type IntegrationTest or DistributedTest
  • Should follow guidelines of either Integration or DistributedTest
  • Should have Javadocs on the test class that reference the bug and describe it sufficiently without having to look up the bug in JIRA
  • May use black-box or white-box testing to guarantee that a bug has been fixed and will not be reintroduced without causing this test to fail

The primary testing frameworks and libraries used in Geode tests (all of the above test categories) are:

  • JUnit 4
  • System Rules – powerful set of JUnit 4 Rules
  • AssertJ – richer than JUnit 4 Assert and includes better support for expected exceptions
  • Awaitility – useful for awaiting asynchronous conditions
  • Mockito – preferred over all other mocking libraries
  • JUnitParams – provides test method parameterization
  • Catch-Exception – better support for expected exceptions especially with complex nesting of exceptions or exceptions with unusual APIs
  • PowerMock – use this as a last resort – refactoring code to facilitate better testing is preferred

Commonly used custom JUnit 4 Rules and testing frameworks that are part of the Geode codebase include:

  • geode-junit JUnit 4 Rules:
    • ExecutorServiceRule – provides ExecutorService for concurrent testing
    • ExpectedTimeoutRule – verifies that an API times out and potentially throws TimeoutException
    • JarFileRule – creates a Jar file in an internal TemporaryFolder containing a dynamically generated class
    • RepeatRule – repeatedly executes a test
    • RequiresGeodeHome – asserts precondition that GEODE_HOME system property is specified
    • RestoreLocaleRule – restores JVM to original Locale
    • RestoreTCCLRule – restores Thread Context Class Loader
    • RetryRule – retries a test until it passes or exceeds specified number of retries
    • TemporaryFileRule – destroys specified files or directories that aren't created in a TemporaryFolder
    • UseJacksonForJsonPathRule – specifies that JsonPath should use Jackson instead of its default JSON library
    • GfshRule – allows use of Gfsh script string to fork a JVM process in an IntegrationTest
  • geode-junit JUnit 4 Rules that extend commonly used JUnit 4 Rules:
    • SerializableTemporaryFolder – allows a TemporaryFolder to be shared across multiple JVMs in a DistributedTest
    • SerializableTestName – allows a TestName to be shared across multiple JVMs in a DistributedTest
  • geode-core (under test) old TestCase classes for DistributedTest:
    • DistributedTestCase – base class for pre-existing distributed tests
    • CacheTestCase – base class for pre-existing distributed tests with APIs for getCache
  • geode-core (under test) API classes for DistributedTest:
    • AsyncInvocation – performs asynchronous invocations in multiple JVMs with Future API
    • Disconnect – disconnects JVMs from Geode cluster
    • DistributedTestUtils – misc utility APIs that don't fit into any of the other DistributedTest API classes
    • DUnitBlackboard – provides a blackboard (distributed map) that can be used in multiple JVMs
    • IgnoredException – specifies Exception string that will be ignored during suspect string grep which occurs in tearDown of DistributedTests
    • Invoke – APIs to invoke SerializableRunnables and SerializableCallables in multiple JVMs
    • NetworkUtils – networking utility APIs to get IP or host name
    • ThreadUtils – utility APIs to dumpt thread stacks
    • VM – the API representation of a remote JVM which is fetched from Host class
  • geode-core (under test) JUnit 4 Rules for DistributedTest:
    • DistributedRule – use this instead of extending DistributedTestCase
    • CacheRule – use this with DistributedRule instead of extending CacheTestCase
    • ClusterStartupRule – use this by itself for Gfsh and Management tests
    • DistributedDisconnectRule – disconnects all JVMs from Geode cluster (similar to APIs in Disconnect class)
    • DistributedRestoreSystemProperites – version of RestoreSystemProperites which executes in all JVMs
    • DistributedUseJacksonForJson – version of UseJacksonForJsonPathRule which executes in all JVMs
    • SharedCountersRule – provides distributed counters and APIs which can be used in all JVMs
    • SharedErrorCollector – version of ErrorCollector which can be used in all JVMs

Always write unit tests for new functionality and always ensure that all unit tests pass before submitting a pull request. Try to make sure new functionality is covered by unit tests, whether or not there are also integration tests as well.

The main build target will execute UnitTests only:

./gradlew build

Our ultimate goal is to have the highest quality product for our users. There are multiple ways of slicing this issue, but if we go all the way back to when the code is being developed, we find unit tests. Unit tests are the foundation in continuous software testing, with the top characteristic of being automated. Good unit testing can save a lot of time and energy later on in the software development lifecycle.

Possible goals to strive for when creating our tests: (from http://artofunittesting.com/definition-of-a-unit-test/)

  • Able to be fully automated
  • Has full control over all the pieces running (Use mocks or stubs to achieve this isolation when needed)
  • Can be run in any order if part of many other tests
  • Runs in memory (no DB or File access, for example)
  • Consistently returns the same result (You always run the same test, so no random numbers, for example. save those for integration or range tests)
  • Runs fast
  • Tests a single logical concept in the system
  • Readable
  • Maintainable
  • Trustworthy (when you see its result, you don't need to debug the code just to be sure)

Good sites for getting ideas:

  • No labels