The dynamic domain reduction procedure for test data generation

Abstract

Test data generation is one of the most technically challenging steps of testing software, but most commercial systems currently incorporate very little automation for this step. This paper presents results from a project that is trying to find ways to incorporate test data generation into practical test processes. The results include a new procedure for automatically generating test data that incorporates ideas from symbolic evaluation, constraint-based testing, and dynamic test data generation. It takes an initial set of values for each input, and dynamically 'pushes' the values through the control-flow graph of the program, modifying the sets of values as branches in the program are taken. The result is usually a set of values for each input parameter that has the property that any choice from the sets will cause the path to be traversed. This procedure uses new analysis techniques, offers improvements over previous research results in constraintbased testing, and combines several steps into one coherent process. The dynamic nature of this procedure yields several benefits. Moving through the control flow graph dynamically allows path constraints to be resolved immediately, which is more efficient both in space and time, and more often successful than constraint-based testing. This new procedure also incorporates an intelligent search technique based on bisection. The dynamic nature of this procedure also allows certain improvements to be made in the handling of arrays, loops, and expressions; language features that are traditionally difficult to handle in test data generation systems. The paper presents the test data generation procedure, examples to explain the working of the procedure, and results from a proof-of-concept implementation. Constraint-based test data generationPrevious work [6,22] presented an approach to test data generation that uses control-flow analysis, symbolic evaluation, and information about mutants to automatically generate test † Note that this definition does not include all test criteria. For example, testing can be based on mean-time-to-failure, which does not use test requirements or a coverage measure. Non-structural criteria are not considered in this research.

An address mapping approach for test data generation of dynamic linked structures

“…A number of automatic test data generation methods have been proposed. Korel [9] classified these methods into three types: random [5,18], pathwise [9,12], and data specification [5,17]. Random testing and data specification are not appropriate for a program with dynamic structure type since the search space is large.…”

“…Pointer variables range from simple structure (single storage) to recursively defined structure (many discrete storage that form shapes such as linked list, tree, heap, etc.). For simple pointer structure such as 'int *a', generating test data can exploit the algorithms of prior work [6,12]. However, some relevant features such as alias and pointer arithmetic must be considered as well.…”

“…If any constraint cannot be satisfied during statement evaluation, the algorithm will report an infeasible path. The second step focuses on data generation which rests primarily on existing data generation methods such as [10,12].…”

“…In this paper, the dynamic domain reduction approach is selected due to its scalability and practical use. Details of the algorithm can be obtained from [12]. In this example, the algorithm will generate numerical data for variables v, p/key, and x/key, assuming that the initial domain for each variable lies between K10 and 10.…”

“…These include program (function), input variable of a function, basic block, predicate, constraint, path, flow graph, control path, feasible path, and infeasible path. Details of these terms are given in [3,12].…”