Extracting QuickCheck specifications from EUnit test cases

Abstract

Writing EUnit tests is more common than writing QuickCheck specifications, although QuickCheck specifications potentially explore far more scenarios than manually written unit tests. In particular for implementations that have side-effects, writing a good set of EUnit tests is often difficult and labour intensive.In this paper we report on mechanisms to extract QuickCheck specifications from EUnit test suites. We use the QSM algorithm to infer state machines from sets of positive and negative traces derived from the test suite. These traces can be derived either statically or dynamically and we describe both approaches here. Finally we show how to move from the inferred state machine to a QuickCheck state machine. This QuickCheck state machine can then be used to generate tests, which include the EUnit tests, but also include many new and different combinations that can augment the test suite. In this way, one can achieve substantially better testing with little extra work.

Model extraction and test generation from JUnit test suites

“…We have also used QSM as the core algorithm for our previous work in test generation [2], but the work presented in this paper differs from it in that we are now combining data dependencies on the model. This addition allows it to convey aspects of the system under test that go beyond the ones learnable by the pure regular inference.…”

“…In this section we discuss in detail the result of applying our model extraction methodology by running our James tool on a frequency server example, as also used in our original work on model extraction for Erlang/EUnit [2]. The fully extracted machine is presented in Fig.…”

“…It simulates a "spectrum management" system that allows clients to allocate and deallocate frequencies while ensuring that each frequency is allocated by at most one client at a time. In [2], we already used the original version of this example for illustrating the tool for transforming EUnit tests into PBT models.…”

“…Assuming that the tests make a sensible exploration of the SUT, then it is possible, not only to construct a graphical model of the system (as shown), but also to construct a QuickCheck Finite State Machine model for the SUT that will generate new tests when executed. We outline how this is done here, building on the approach first presented in [2].…”

“…In doing this we follow earlier work [2] for Erlang in which finite-state machine models and properties were extracted from EUnit test suites.…”