Guided path exploration for regression test generation

Abstract

Regression test generation aims at generating a test suite that can detect behavioral differences between the original and the modified versions of a program. Regression test generation can be automated by using Dynamic Symbolic Execution (DSE), a state-of-the-art test generation technique, to generate a test suite achieving high structural coverage. DSE explores paths in the program to achieve high structural coverage, and exploration of all these paths can often be expensive. However, if our aim is to detect behavioral differences between two versions of a program, we do not need to explore all paths in the program as not all these paths are relevant for detecting behavioral differences. In this paper, we propose a guided path exploration approach that avoids exploring irrelevant paths and gives priority to more promising paths (in terms of detecting behavioral differences) such that behavioral differences are more likely to be detected earlier in path exploration. Preliminary results show that our approach requires about 12.9% fewer runs on average (maximum 25%) to cause the execution of a changed statement and 11.8% fewer runs on average (maximum 31.2%) to cause program-state differences after its execution than the search strategies without guidance.

An Effective Regression Testing Approach for PHP Web Applications

“…However, web applications involve different challenges than desktop applications written in C or Java [10], and the majority of web applications heavily deal with strings in addition to numeric values. In addition, while Taneja et al [8] propose an efficient test generation technique by pruning unnecessary paths, the dynamic symbolic execution-based test generation approach used by other researchers can still be expensive and infeasible when we apply it to large size applications.…”

“…Creating new, executable test cases for affected areas of the code, which is known as a test suite augmentation problem [6], is one of the important regression testing problems, but only a few researchers have started working on this problem [6], [7], [8], [9]. While their work has made some progress in this area, they have only provided guidance for creating new tests [7], generated new test cases limited to numeric values [9], and considered only small desktop applications.…”

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

“…For example, the research community has invested a lot of effort in designing techniques for improving the testing of software patches, ranging from test suite prioritisation and selection algorithms [11,30,35] to program analysis techniques for test case generation and bug finding [1,2,20,21,27,28,36,40] to methods for surviving errors introduced by patches at runtime [14]. Many of these techniques depend on the existence of a manual test suite, sometimes requiring the availability of a test exercising the patch [24,37], sometimes making assumptions about the stability of program coverage or external behaviour over time [14,29], other times using it as a starting point for exploration [10,16,22,39], and often times employing it as a baseline for comparison [3,6,9,26].…”

High-Coverage Symbolic Patch Testing

“…Synthesizing inputs which cover a target is an essential problem in automated test generation and debugging [1,16,27,29,30,32]. While we borrow ideas from the state of the art in these areas, and combine symbolic execution, static analysis and various heuristics, our approach differs by treating the task as an optimization problem with the goal of exploring paths that minimize the estimated distance to the target.…”

“…eXpress [27] improves on directed test suite augmentation by pruning CFG branches which provably do not lead to the patch. While eXpress does not depend on having existing test cases that reach the source node of an uncovered branch and its algorithm allows it to prune significant parts of the search space, it does not actively try to steer execution toward the patched code.…”