Towards an Improvement of Bisection-Based Adaptive Random Testing

Mao, Chengying; Zhan, Xuzheng

doi:10.1109/apsec.2017.86

Cited by 9 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the existing ART algorithms, a typical strategy to reduce the cost of distance computation is to partition the input domain so that each candidate only needs to compute the distance to some of the executed test cases. For numerical input domains, they usually can be subdivided into some regular forms, such as grid partitioning, 36,38 bisection, [39][40][41] etc. However, when dealing with complex data types, it becomes challenging to clearly partition their corresponding input domains using expressions based on input parameters.…”

Section: Overall Frameworkmentioning

confidence: 99%

A lightweight adaptive random testing method for deep learning systems

Mao,

Song,

Chen

2023

Softw Pract Exp

Self Cite

View full text Add to dashboard Cite

In recent years, deep learning (DL) systems are increasingly used in the safety‐critical fields such as autonomous driving, medical diagnosis, and financial service. Although these systems have demonstrated an outstanding performance in enhancing the accuracy of decision‐making, they pose significant challenges to the trustworthiness due to their limited interpretability and inherent uncertainty. Adaptive random testing (ART) has been proved as an effective approach for ensuring the reliability of DL systems. However, existing ART methods for DL systems incur a heavy overhead in test case selection due to the computation of distances. To address this issue, we propose a lightweight adaptive random testing (Lw‐ARTDL) method for DL systems. In our improved algorithm, we employ the K‐Means technique to divide the entire test suite into several subsets. Then, for a candidate test case, we only calculate distances between it and the test cases within the category to which it belongs. This partition strategy ensures that the selected test cases are more representative while significantly reducing the computational cost. To validate the proposed algorithm, the comparison experiments between Lw‐ARTDL and the original ARTDL algorithm are conducted on two typical DL systems. The experimental results show that Lw‐ARTDL significantly reduces the overhead of failure detection, and exhibits stronger failure detection capability compared to ARTDL in most similarity metrics.

show abstract

Section: Overall Frameworkmentioning

confidence: 99%

A lightweight adaptive random testing method for deep learning systems

Mao,

Song,

Chen

2023

Softw Pract Exp

Self Cite

View full text Add to dashboard Cite

show abstract

“…Mayer [94] used a similar mechanism to improve the effectiveness of PBS with bisection partitioning. Mao and Zhan [95] also used this mechanism to enhance PBS by bisection partitioning, but instead of Euclidean distance, they used the coordinate distance to boundaries (boundary distance). Similarly, Mayer [96], [97], for PBS with random and bisection partitioning, used exclusion regions in a possible subdomain to generate a new test case.…”

Section: Stfcs + Pbsmentioning

confidence: 99%

A Survey on Adaptive Random Testing

Huang

Sun

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Random testing (RT) is a well-studied testing method that has been widely applied to the testing of many applications, including embedded software systems, SQL database systems, and Android applications. Adaptive random testing (ART) aims to enhance RT's failure-detection ability by more evenly spreading the test cases over the input domain. Since its introduction in 2001, there have been many contributions to the development of ART, including various approaches, implementations, assessment and evaluation methods, and applications. This paper provides a comprehensive survey on ART, classifying techniques, summarizing application areas, and analyzing experimental evaluations. This paper also addresses some misconceptions about ART, and identifies open research challenges to be further investigated in the future work.

show abstract

“…Since the test cases in each sub‐domain are generated randomly, it is difficult for B‐ART to avoid that the test cases in two adjacent sub‐domains are too close. Accordingly, B‐ART based on flexible partitioning and candidate strategy algorithm [10 ] is proposed to overcome this problem. In the RP‐ART, the input domain (or sub‐domain) is divided according to the coordinates of the currently selected test case.…”

Section: Related Workmentioning

confidence: 99%

“…At present, the typical methods include the ART by bisection (B‐ART) and the ART by random partitioning (RP‐ART) [7 ]. Experiments show that this kind of method can produce a set of test cases with significantly better fault‐revealing ability than RT at a small‐time cost, but compared with the fixed‐size‐candidate‐set ART (FSCS‐ART) [3, 8 ], its failure‐detection ability still needs to be further improved [9, 10 ].…”

Section: Introductionmentioning

confidence: 99%

Adaptive random testing based on flexible partitioning

et al. 2020

Self Cite

View full text Add to dashboard Cite

Adaptive random testing (ART) achieves better failure‐detection effectiveness than random testing due to its even spreading of test cases. ART by random partitioning (RP‐ART) is a lightweight method, but its advantage over random testing is relatively low. Although iterative partition testing (IPT) method has good performance for detecting failures in a block pattern, it loses randomness during the test case generation. To overcome the shortcomings of the above two algorithms, a new algorithm named ART by flexible partitioning (FP‐ART) is proposed. In the FP‐ART, a set of random candidates is used to select an appropriate test case by considering their boundary distance. Accordingly, the corresponding sub‐domain is also partitioned by the new test case. Based on this kind of flexible partitioning, the randomness of test case selection can be guaranteed and the spatial distribution of test cases is even more diverse. According to the results in simulation and empirical experiments, FP‐ART demonstrates better failure‐detection effectiveness than RP‐ART and is more suitable to detect the failures in strip patterns than the IPT method. Meanwhile, its failure‐detection ability is much stronger than that of fixed‐size‐candidate‐set ART in the cases of a relatively high failure rate.

show abstract

Towards an Improvement of Bisection-Based Adaptive Random Testing

Cited by 9 publications

References 19 publications

A lightweight adaptive random testing method for deep learning systems

A lightweight adaptive random testing method for deep learning systems

A Survey on Adaptive Random Testing

Adaptive random testing based on flexible partitioning

Contact Info

Product

Resources

About