Applying SMT Solvers to the Test Template Framework

Abstract

The Test Template Framework (TTF) is a model-based testing method for the Z notation. In the TTF, test cases are generated from test specifications, which are predicates written in Z. In turn, the Z notation is based on first-order logic with equality and Zermelo-Fraenkel set theory. In this way, a test case is a witness satisfying a formula in that theory. Satisfiability Modulo Theory (SMT) solvers are software tools that decide the satisfiability of arbitrary formulas in a large number of built-in logical theories and their combination. In this paper, we present the first results of applying two SMT solvers, Yices and CVC3, as the engines to find test cases from TTF's test specifications. In doing so, shallow embeddings of a significant portion of the Z notation into the input languages of Yices and CVC3 are provided, given that they do not directly support Zermelo-Fraenkel set theory as defined in Z. Finally, the results of applying these embeddings to a number of test specifications of eight cases studies are analysed.

A family of simulation criteria to guide DEVS models validation rigorously, systematically and semi-automatically

“…Further, free variables range over infinite sets, making the problem undecidable. One possibility is to adopt a Satisfiability Modulo Theories (SMT) solver [37,38], by adapting the work of Cristiá and Frydman [39]. Another alternative could be to use constraint solvers such as {log} (pronounced 'setlog') [40,41,42].…”

Tool support for the Test Template Framework

“…Although the TTF has been extended to deal with axiomatic definitions and first‐order quantifiers , further research is needed. A medium‐term goal is to study the chances to integrate Fastest with a SMT solver with the intention to improve the generation of abstract test cases . Finally, the authors are currently working on the other steps of Figure , in particular on abstract test case refinement and execution.…”

“…For instance, the most obvious solution would have been to rely on an SMT solver , but the following were found: (i) it was impossible to find an existing SMT solver covering all or a significant portion of the ZMT; and (ii) either implementing an SMT solver from scratch or trying to codify the ZMT in an existing SMT is a research project on its own not directly related with MBT. However, recently, Cristiá and Frydman have been experimenting with SMT solvers .…”

“…Such SMT solvers include, for instance, Z3 , Yices and CVC3 . Cristiá and Frydman applied Yices and CVC3 to 69 satisfiable TTF test specifications . Other approaches solve similar problems but in different contexts, making them less useful for the TTF.…”