Improving deep‐learning‐based fault localization with resampling

Abstract

Many fault localization approaches recently utilize deep learning to learn an effective localization model showing a fresh perspective with promising results. However, localization models are generally learned from class imbalance datasets; that is, the number of failing test cases is much fewer than passing test cases. It may be highly susceptible to affect the accuracy of learned localization models. Thus, in this paper, we explore using data resampling to reduce the negative effect of the imbalanced class problem and improve the accuracy of learned models of deep‐learning‐based fault localization. Specifically, for deep‐learning‐based fault localization, its learning feature may require duplicate essential data to enhance the weak but beneficial experience incurred by the class imbalance datasets. We leverage the property of test cases (i.e., passing or failing) to identify failing test cases as the duplicate essential data and propose an iterative oversampling approach to resample failing test cases for producing a class balanced test suite. We apply the test case resampling to representative localization models using deep learning. Our empirical results on eight large‐sized programs with real faults and four large‐sized programs with seeded faults show that the test case resampling significantly improves fault localization effectiveness.

CRMF: A fault localization approach based on class reduction and method call frequency

“…In this study, we use three metrics to evaluate the fault localization effectiveness of the techniques: $$$ EXAM $$$ score ($$$ EXAM $$$), Accuracy ($$$ acc@n $$$), $$$ MRR $$$, and Wilcoxon signed‐rank test. All these metrics have been widely used in previous fault localization studies 7,20,46,55‐57 …”

“…All these metrics have been widely used in previous fault localization studies. 7,20,46,[55][56][57] 4.4.1 EXAM score (EXAM) EXAM score 7,55 is the proportion of program elements, which are ranked before the actual faulty element in the suspiciousness rank lists. It is a commonly used metric for fault localization techniques, and a lower EXAM score value indicates a better performance of the corresponding fault localization technique.…”

MetaFL: Metamorphic fault localisation using weakly supervised deep learning

“…Mutants that have a failure rate of 10%~90% in all four scenarios (described below) are accepted as valid mutants. The rest are filtered out to mitigate the impact of class imbalance problem [44] to our experiments, and we consider it out of scope of this paper to adopt techniques such as resampling [45] for handling this problem. The numbers of mutants for each program are shown in Table 2.…”

A Learning-Based Fault Localization Approach Using Subset of Likely and Dynamic Invariants

“…Zhang et al [31] investigate the use of dataset resampling to mitigate the negative effects of the imbalanced class dilemma and improves the precision of the deep-learning-based fault localization proposed model in this study. Deep-learning-based fault localization, in particular, may necessitate duplicate vital data to improve the weaker but useful experience caused by the imbalanced class collections.…”

“…The next section presents the details of this approach. [29] 2021 Assessment based approach Lou et al [30] 2020 Supervised/unsupervised learning Zhang et al [31] 2021 Oversampling, Neural networks Peng et al [32] 2020 Auto-encoders Li et al [33] 2021 Image pattern recognition Bartocci et al [34] 2021 Cyber physical systems Assi et al [35] 2021 Spectrum based approach Wardat et al [36] 2021 MLP Zhang et al [37] 2021 CNN, RNN, MLP…”