A concept analysis inspired greedy algorithm for test suite minimization

Abstract

Software testing and retesting occurs continuously during the software development lifecycle to detect errors as early as possible and to ensure that changes to existing software do not break the software. Test suites once developed are reused and updated frequently as the software evolves. As a result, some test cases in the test suite may become redundant as the software is modified over time since the requirements covered by them are also covered by other test cases. Due to the resource and time constraints for re-executing large test suites, it is important to develop techniques to minimize available test suites by removing redundant test cases. In general, the test suite minimization problem is NP complete. In this paper, we present a new greedy heuristic algorithm for selecting a minimal subset of a test suite T that covers all the requirements covered by T . We show how our algorithm was inspired by the concept analysis framework. We conducted experiments to measure the extent of test suite reduction obtained by our algorithm and prior heuristics for test suite minimization. In our experiments, our algorithm always selected same size or smaller size test suite than that selected by prior heuristics and had comparable time performance.

Regression testing minimization, selection and prioritization: a survey

“…Tallam and Gupta developed the greedy approach further by introducing the delayed greedy approach, which is based on the Formal Concept Analysis of the relation between test cases and testing requirements 18. A potential weakness of the greedy approach is that the early selection made by the greedy algorithm can eventually be rendered redundant by the test cases subsequently selected.…”

“…A potential weakness of the greedy approach is that the early selection made by the greedy algorithm can eventually be rendered redundant by the test cases subsequently selected. For example, consider the test suite and testing requirements depicted in Table I, taken from Tallam and Gupta 18. The greedy approach will select t 1 first as it satisfies the maximum number of testing requirements, and then continues to select t 2 , t 3 and t 4 .…”

“…While minimization and prior‐itization techniques lack such a framework, certain metrics have been used as a de facto standard evaluation framework. Rate of reduction in size and rate of reduction in fault‐detection capability have been widely used to evaluate test suite minimization techniques 12, 18–21, 25, 26, 38–40, 43. Similarly, APFD 116 has been widely used to evaluate prioritization techniques 23, 116, 117, 120, 121, 123, 127, 136–138, 145, 146, 154, 156, 157, 159, 160, 162–165.…”

Feature interaction analysis

“…[16]), due to (1) its programming language and application domain independence that allows users to easily define different views (using different object, attribute combinations); (2) its availability of tool support to automatically generate context tables and lattices; (3) the fact that it is a relatively inexpensive analysis, in particular compared to other more traditional dynamic dependency and trace analysis techniques. However, it has to be noted that FCA does not consider semantic information during the analysis step, limiting its applicability for certain analysis tasks.…”

Reduction Method of Regression Testing Requirements Based on Relation Mode

“…This method analyzes and calculates the back node of decision graph, then obtains the inclusion relationship among the testing requirements to calculate the expansion, and ultimately reduces the size of the test suite. S. Tallam et al considered both testing requirement reduction and test suite optimization, and proposed the reduction method of the concept analysis [3]. The mentioned methods above mainly reduce the inclusion relation, and it should be continued to study how to deal with the relationship among the testing requirements.…”