Random testing

Gerlich, R. Nicholas; Boll, Thomas J.

doi:10.1145/1292414.1292424

Cited by 11 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, a number of studies have shown that there are a number of vulnerabilities in some mainstream frameworks of DL or their applications, [30][31][32] and these potential faults bring a potential threat to the reliability of DL systems. Since the learning process of DL systems is often not clear enough, black-box testing methods represented by random testing 12 and fuzzy testing 33 are preferred for testing such systems. Recently, Yan et al 22 first employed ART to detect failures of DL systems.…”

Section: Review Of Artdl Algorithmmentioning

confidence: 99%

“…For systems with typical black-box characteristics like DL systems, the black-box testing method is the most natural and least costly solution for ensuring their quality assurance. Among all black-box testing methods, random testing 12 stands out as a fundamental approach, characterized by its simplicity, high level of automation, and ease of implementation. As a result, it is widely employed in the industry for testing practical software systems.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A lightweight adaptive random testing method for deep learning systems

Mao,

Song,

Chen

2023

Softw Pract Exp

View full text Add to dashboard Cite

In recent years, deep learning (DL) systems are increasingly used in the safety‐critical fields such as autonomous driving, medical diagnosis, and financial service. Although these systems have demonstrated an outstanding performance in enhancing the accuracy of decision‐making, they pose significant challenges to the trustworthiness due to their limited interpretability and inherent uncertainty. Adaptive random testing (ART) has been proved as an effective approach for ensuring the reliability of DL systems. However, existing ART methods for DL systems incur a heavy overhead in test case selection due to the computation of distances. To address this issue, we propose a lightweight adaptive random testing (Lw‐ARTDL) method for DL systems. In our improved algorithm, we employ the K‐Means technique to divide the entire test suite into several subsets. Then, for a candidate test case, we only calculate distances between it and the test cases within the category to which it belongs. This partition strategy ensures that the selected test cases are more representative while significantly reducing the computational cost. To validate the proposed algorithm, the comparison experiments between Lw‐ARTDL and the original ARTDL algorithm are conducted on two typical DL systems. The experimental results show that Lw‐ARTDL significantly reduces the overhead of failure detection, and exhibits stronger failure detection capability compared to ARTDL in most similarity metrics.

show abstract