Selecting Test Cases based on Similarity of Runtime Information: A Case Study of an Industrial Simulator

Shimari, Kazumasa; Tanaka, M.; Ishio, Takashi; Matsushita, Makoto; Inoue, Katsuro; Takanezawa, Satoru

doi:10.1109/icsme55016.2022.00077

Cited by 1 publication

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…73 By observing the logical structure elements or program states covered by test cases, it is found that there are multiple test cases covering the same target. Then, in regression testing scenarios, in order to save testing cost, the previous test cases are often clustered based on their execution profiles 74 or runtime information, 75 and then a subset of them is selected for the next round of testing. Among them, K-Means is the most popular clustering technique used for test case reduction, 76 and other algorithms such as expectation maximization (EM) 77 and evolutionary clustering 78 are also adopted.…”

Section: Applications Of Cluster Analysis In Software Testingmentioning

confidence: 99%

A lightweight adaptive random testing method for deep learning systems

Mao,

Song,

Chen

2023

Softw Pract Exp

View full text Add to dashboard Cite

In recent years, deep learning (DL) systems are increasingly used in the safety‐critical fields such as autonomous driving, medical diagnosis, and financial service. Although these systems have demonstrated an outstanding performance in enhancing the accuracy of decision‐making, they pose significant challenges to the trustworthiness due to their limited interpretability and inherent uncertainty. Adaptive random testing (ART) has been proved as an effective approach for ensuring the reliability of DL systems. However, existing ART methods for DL systems incur a heavy overhead in test case selection due to the computation of distances. To address this issue, we propose a lightweight adaptive random testing (Lw‐ARTDL) method for DL systems. In our improved algorithm, we employ the K‐Means technique to divide the entire test suite into several subsets. Then, for a candidate test case, we only calculate distances between it and the test cases within the category to which it belongs. This partition strategy ensures that the selected test cases are more representative while significantly reducing the computational cost. To validate the proposed algorithm, the comparison experiments between Lw‐ARTDL and the original ARTDL algorithm are conducted on two typical DL systems. The experimental results show that Lw‐ARTDL significantly reduces the overhead of failure detection, and exhibits stronger failure detection capability compared to ARTDL in most similarity metrics.

show abstract