Enhancing mirror adaptive random testing through dynamic partitioning

Abstract

a b s t r a c tContext: Adaptive random testing (ART), originally proposed as an enhancement of random testing, is often criticized for the high computation overhead of many ART algorithms. Mirror ART (MART) is a novel approach that can be generally applied to improve the efficiency of various ART algorithms based on the combination of ''divide-and-conquer'' and ''heuristic'' strategies. Objective: The computation overhead of the existing MART methods is actually on the same order of magnitude as that of the original ART algorithms. In this paper, we aim to further decrease the order of computation overhead for MART. Method: We conjecture that the mirroring scheme in MART should be dynamic instead of static to deliver a higher efficiency. We thus propose a new approach, namely dynamic mirror ART (DMART), which incrementally partitions the input domain and adopts new mirror functions. Results: Our simulations demonstrate that the new DMART approach delivers comparable failure-detection effectiveness as the original MART and ART algorithms while having much lower computation overhead. The experimental studies further show that the new approach also delivers a better and more reliable performance on programs with failure-unrelated parameters. Conclusion: In general, DMART is much more cost-effective than MART. Since its mirroring scheme is independent of concrete ART algorithms, DMART can be generally applied to improve the cost-effectiveness of various ART algorithms.

A Survey on Adaptive Random Testing

“…Previous studies [55]- [57] have combined STFCS with static partitioning, using the concept of mirroring to reduce the computational costs. Enhancements to mirroring have included a revised distance metric [91], and dynamic partitioning with new mirroring functions [92]. Chan et al [47] applied bisection partitioning to each dimension of the input domain, then checking the amount of executed test cases in each subdomain: candidates in subdomains with fewer executed test cases were then more likely to be selected.…”

“…Because ART involves additional computation to evenly spread the test cases over the input domain [17], [18], ART should naturally take more time than RT to generate the same number of test cases, suggesting that it may have worse cost-effectiveness than RT. However, studies [18], [92], [114]- [116] have shown that compared with RT, ART typically requires less time to identify the first failure (F-time) -therefore, ART can be more cost-effective than RT. In general, three main conditions can result in ART achieving a better cost-effectiveness than RT: (a) ART using fewer test cases than RT to detect the first failure (F-measure); (b) the computational overhead of the ART approach being acceptable (comparable or slightly higher than that of RT); or (c) the combined program execution and test setup time being more than the time required by ART to generate a test case.…”

“…Mirroring [55]- [57], [92] has been used to directly generate mirror test cases of a source test case based on a principle of symmetry of subdomains. Although the time complexity for FSCS mirroring can still be O( n 2 m 2 ) [55]- [57] (where m is the number of subdomains), an enhanced mirroring technique can reduce this to O(n) [92]. Shahbazi et al [73] have also proposed a linear-order (O(n)) ART approach: a fast search algorithm for RBCVT (RBCVT-Fast).…”

Elimination by Linear Association: An Effective and Efficient Static Mirror Adaptive Random Testing

“…It can be observed that each test case have similar values along the line of mirror dimensions. Huang et al [28] have pointed out that, irrespective of the number of mirror domains implemented in MART, mirroring functions cannot guarantee diversity on all coordinates of mirror test cases. Hence, this is an inherent problem with the use of mapping function of MART.…”

“…We selected programs with at most four (4) dimensions because the number of MART schemes will be too many and time consuming to analyze if the input dimension is too high. Also, in order to put the performance of EMART into proper perspectives, we adopted twelve (12) programs which have been used extensively [12], [19], [28], [33], [34] in the past to assess the performance of most ART algorithms. These programs were originally written in Fortran Pascal or C; designed for numerical computations and have been converted to C++ programs [6].…”