Automated generation of oracled test cases with regular expressions and combinatorial techniques

Abstract

One of the main challenges of software testing research is the automated addition of oracles to the generated test cases: Whereas the automated generation of operation sequences (which is one of the essential components of test cases) is in practice a solved problem, the automated addition of the oracle (another indispensable element) is still an important problem and an open research question. This article proposes an approach to get executable test suites composed by complete test cases (i.e., they include the oracle). The core of the method is based on annotated regular expressions. The test generation process, which is supported by a tool, follows three steps: (1) creation of annotated regular expressions, where each regular expression describes a set of sequences of operations to be executed against the system under test; (2) expansion of the regular expressions to get sequences of operations, which still do not have parameter values; and (3) generation of the executable test cases with oracle. In this third step, each test case is generated with the suitable oracle, depending on the conditions specified in the regular expression.