...
Before commit, there must exist tests covering critical and deterministically triggered behaviours, for the complete matrix of all connection methods and messaging versions. The goal here is to have extremely quick tests that can provide prompt feedback to developers that may have made a simple error when modifying the semantics of the system.
While we do not explicitly value code coverage metrics, we will produce a code coverage report of this work, and use it to ensure we have not missed any critical behaviours.
Out of scope is any concurrent or non-deterministic behaviours, as well as provider-specific exception handling, and interactions with.
Randomised Testing
It is impossible to fully exercise a sophisticated concurrent system without stress testing it against a model. We will build a burn test that can be left running indefinitely, with the guarantee that it will eventually explore every possible logical system state. It will, at a minimum, simultaneously model all scenarios of:
- Overload, expiry and corruption
- Categories of failure to send or receive (de/serialize, write to wire, etc)
- Graceful and abrupt disconnects triggered by either the application or the provider
- Arbitrarily slow:
- Connections
- Message payload processing
- Message de/serialization
- Message submission by application
- Resource consumption
Verification entails confirming that in allof these scenarios, in any interleaving, the system behaviour is precisely as specified, i.e. that
...