KDFC-ART: a KD-tree approach to enhancing Fixed-size-Candidate-set Adaptive Random Testing

Mao, Chengying; Zhan, Xuzheng; Tse, T. H.; Chen, Tsong Yueh

doi:10.1109/tr.2019.2892230

Cited by 20 publications

(23 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the original FSCS-ART uses exact NNS, it has been found that an ANNS can also be employed that maintains the FSCS-ART failure-detection effectiveness [46].…”

Section: Fixed-size-candidate-set Adaptive Random Testingmentioning

confidence: 99%

“…A state-of-the-art FSCS-ART overhead reduction strategy called KDFC-ART [46] stores previously executed, nonfailure-causing test cases in a tree-based data structure to perform efficient NNS. LimBal-KDFC -the most efficient of the three KDFC-ART variants -incorporates limited backtracking and a semi-balancing strategy to perform ANNS, and appears to effectively address high-dimensionality computational challenges.…”

Section: State-of-the-art: Kdfc-artmentioning

confidence: 99%

“…The programs pntLinePos, pntTriangle-Pos, twoLinesPos and triangle were written as exercises from the textbook Introduction to Java Programing and Data Structures [89]. The nearestDistance program takes five points in 2-dimensional space and returns the two points that are nearest to each other [46]. The calGCD program takes 10 integers and returns their greatest common divisor.…”

Section: Simulations and Subject Programsmentioning

confidence: 99%

“…We used T G and T E to evaluate the efficiency of the methods (RQ1). Although F-time (the sum of the generation, execution, and evaluation times for a test case) [106] has also been used in some ART studies, reporting the component times separately appears to be the preferred option when evaluating FSCS-ART computational overhead reduction strategies [107,108,46] this removes ambiguity in scenarios where a method may have less T G but more T E .…”

Section: Construct Validitymentioning

confidence: 99%

“…In software testing, NNS has been used to find the most similar test cases in regression testing [43], test case prioritization (TCP) [44], and model-based testing [45]. It has also been used to find the most diverse (opposite to similar) test cases in ART [46] and software product lines [47]. ANNS has also been successfully applied to enhance the efficiency in other areas of software testing, including TCP [44], test suite reduction [48] and prediction of test flakiness [49].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

SWFC-ART: A Cost-effective Approach for Fixed-Size-Candidate-Set Adaptive Random Testing through Small World Graphs

Ashfaq,

Huang,

Towey

et al. 2021

Preprint

View full text Add to dashboard Cite

Adaptive random testing (ART) improves the failure-detection effectiveness of random testing by leveraging properties of the clustering of failure-causing inputs of most faulty programs: ART uses a sampling mechanism that evenly spreads test cases within a software's input domain. The widely-used Fixed-Sized-Candidate-Set ART (FSCS-ART) sampling strategy faces a quadratic time cost, which worsens as the dimensionality of the software input domain increases. In this paper, we propose an approach based on small world graphs that can enhance the computational efficiency of FSCS-ART: SWFC-ART. To efficiently perform nearest neighbor queries for candidate test cases, SWFC-ART incrementally constructs a hierarchical navigable small world graph for previously executed, non-failure-causing test cases. Moreover, SWFC-ART has shown consistency in programs with high dimensional input domains. Our simulation and empirical studies show that SWFC-ART reduces the computational overhead of FSCS-ART from quadratic to log-linear order while maintaining the failure-detection effectiveness of FSCS-ART, and remaining consistent in high dimensional input domains. We recommend using SWFC-ART in practical software testing scenarios, where real-life programs often have high dimensional input domains and low failure rates.

show abstract

“…Although the original FSCS-ART uses exact NNS, it has been found that an ANNS can also be employed that maintains the FSCS-ART failure-detection effectiveness [46].…”

Section: Fixed-size-candidate-set Adaptive Random Testingmentioning

confidence: 99%

Section: State-of-the-art: Kdfc-artmentioning

confidence: 99%

Section: Simulations and Subject Programsmentioning

confidence: 99%

Section: Construct Validitymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

SWFC-ART: A Cost-effective Approach for Fixed-Size-Candidate-Set Adaptive Random Testing through Small World Graphs

Ashfaq,

Huang,

Towey

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

A lightweight adaptive random testing method for deep learning systems

Mao,

Song,

Chen

2023

Softw Pract Exp

Self Cite

View full text Add to dashboard Cite

In recent years, deep learning (DL) systems are increasingly used in the safety‐critical fields such as autonomous driving, medical diagnosis, and financial service. Although these systems have demonstrated an outstanding performance in enhancing the accuracy of decision‐making, they pose significant challenges to the trustworthiness due to their limited interpretability and inherent uncertainty. Adaptive random testing (ART) has been proved as an effective approach for ensuring the reliability of DL systems. However, existing ART methods for DL systems incur a heavy overhead in test case selection due to the computation of distances. To address this issue, we propose a lightweight adaptive random testing (Lw‐ARTDL) method for DL systems. In our improved algorithm, we employ the K‐Means technique to divide the entire test suite into several subsets. Then, for a candidate test case, we only calculate distances between it and the test cases within the category to which it belongs. This partition strategy ensures that the selected test cases are more representative while significantly reducing the computational cost. To validate the proposed algorithm, the comparison experiments between Lw‐ARTDL and the original ARTDL algorithm are conducted on two typical DL systems. The experimental results show that Lw‐ARTDL significantly reduces the overhead of failure detection, and exhibits stronger failure detection capability compared to ARTDL in most similarity metrics.

show abstract