In this paper we present a case study of applying combinatorial testing to test a combinatorial test generation tool called ACTS. The purpose of this study is two-fold. First, we want to gain experience and insights about how to apply combinatorial testing in practice. Second, we want to evaluate the effectiveness of combinatorial testing applied to a real-life system. ACTS has 24637 lines of uncommented code, and provides a command line interface and a fairly sophisticated graphic user interface. The main challenge of this study was to model the input space in terms of a set of parameters and values. Once the model was designed, we generated test cases using ACTS, which were then later used to test ACTS. The results of this study show that input space modeling can be a significant undertaking, and needs to be carefully managed. The results also show that combinatorial testing is effective in terms of achieving high code coverage and fault detection.
Combinatorial testing has been shown to be a very effective testing strategy. An important problem in combinatorial testing is dealing with constraints, i.e., restrictions that must be satisfied in order for a test to be valid. In this paper, we present an efficient algorithm, called IPOG-C, for constraint handling in combinatorial testing. Algorithm IPOG-C modifies an existing combinatorial test generation algorithm called IPOG to support constraints. The major contribution of algorithm IPOG-C is that it includes three optimizations to improve the performance of constraint handling. These optimizations can be generalized to other combinatorial test generation algorithms. We implemented algorithm IPOG-C in a combinatorial test generation tool called ACTS. We report experimental results that demonstrate the effectiveness of algorithm IPOG-C. The three optimizations increased the performance by one or two orders of magnitude for most subject systems in our experiments. Furthermore, a comparison of ACTS to three other tools suggests that ACTS can perform significantly better for systems with more complex constraints.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.