Apache Ignite uses TeamCity (TC) to provide continious integration and unit/integration testing. Server is available at http://ci.ignite.apache.org/
TC contains several projects for different Apache Ignite version(s), actual versions development is done in Ignite Tests 2.4+ (Java 8)
Previous version test projects:
Ignite is tested using approximate 90 build configurations (in terms of TC) for testing. Each build configuration has 1 or more (root) test suites to start. These suites group tests around Ignite feature and/or module. Each (root) test suite has a lot of suites (JUnit), each JUnit suite consist from tests.
Example of full test naming:
where IgniteStandByClusterSuite - grouping suite, IgniteStandByClusterSuite - JUnit suite, IgniteStandByClusterSuite - test method.
Build configurations can be started individually or using build chain.
Build chains are used like composite project to summarize results of each run configuration results.
There are several chains for testing different features in Ignite (Persistent Data Store, SQL, .NET, CPP etc)
There are also 2 common chains. Run All includes all actual valid tests to be executed for PR checks, also Run All is triggered by schedule triggers for master and release branches.
Always use Run All chain to check PR before merge
Run Basic is subset of RunAll, but includes fastest and stable tests to be used to check each commit to master
For Run All Basic chain VCS triggers are enabled.
There is EMail notification send if contibutor changes were included in build which failed.
Make sure your TeamCity user is correctly matched with github user
Approximate parameters for tests to be suitable to each group can be found in table below.
There is Make Teamcity Green Again process running in the Ignite community to make following target parameters better.
|Build Configuration Max execution time||Test Execution Time||Fail Rate for suite||Fail Rate For Test|
|Run All Basic||30 min||< 3 %||< 1%|
|Run All||2h||< 3 min||< 30%||< 10%|