Computing reliability: On the differences between software testing and software fault injection techniques

Kooli, Maha; Kaddachi, Firas; Natale, Giorgio Di; Bosio, Alberto; Benoit, Pascal; Torres, Lionel

doi:10.1016/j.micpro.2017.02.007

Cited by 21 publications

(10 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our study, we use M T P as the mutation reduction performance metric and EXAM score as the fault localization accuracy metric. The former metric is popularly used in evaluating the cost of MBFL [12], [21], [32], and the latter one is also a commonly used fault localization metric in previous studies [45], [46]. Therefore, the performance metrics used in our study can reflect the real situation in evaluating MFL methods.…”

Section: Threats To Validitymentioning

confidence: 99%

“…Mutation-based fault localization (MBFL) technique is based on mutation analysis [12], and its main foundation is that a mutant can be considered as a similar version or a partial fix of real faults [2]. As shown in Figure 1, MBFL has the following four steps: 1) Obtain statements covered by failed test cases: given a program P with n statements and a test suite T with m test cases, MBFL will execute P against T and collect the coverage information and test results (i.e., pass or fail).…”

Section: B Mutation-based Fault Localizationmentioning

confidence: 99%

“…Therefore, a total of 112 faulty versions are used in our experiments. In our study, we use the GNU gcov tool [39] to collect coverage information and Proteum tool [40] to generate mutants, both of these tools are popular and have been widely adopted by many researchers [12], [21], [32]. Besides, we employ mutation operators suggested by the work of Agrawal et al [41].…”

Section: B Subject Programsmentioning

confidence: 99%

“…The main threat is the mutation tools used in our experiments. We choose Proteum for mutation analysis, which is a popular mutation analysis tool and is widely used in previous studies [12], [13], [21], [32].…”

Section: Threats To Validitymentioning

confidence: 99%

“…MBFL techniques have higher fault localization accuracy, but their efficiency is suffered from an extremely mutant execution cost problem [12], [16], which results in the lower efficiency of fault localization. This problem stems from that MBFL generates a significant number of mutants from the program under test, and each mutant should be executed on all test cases.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Mutation-based fault localization (MBFL) is a recently proposed technique with the advantage of high fault localization accuracy. However, such a mutation analysis based technique is difficult to be accepted by industry due to its huge computational cost on mutation analysis. There are three ways to improve MBFL's efficiency, which are reducing the number of mutants, optimizing the mutants' execution process, and reducing the number of test cases. The former two ways have been mainly studied and shown promising results, but for the latter way, the related studies are limited since this kind of method will reduce the precision of MBFL. In this paper, we mainly focus on the latter way and propose an information entropy based test case reduction (IETCR) strategy for MBFL. In particular, we first calculate the entropy change of test cases and select a proportion of them according to their value. Then we use a reduced test suite to execute mutants. To show the effectiveness of the IETCR strategy, we choose six real-world programs with faulty versions. In terms of mutation reduction rate, we find MBFL with the IETCR strategy can reduce 56.3%∼88.3% cost while keeping almost the same fault localization accuracy when compared to the original MBFL without test case reduction. Moreover, we use Wilcoxon signed-rank test for statistical analysis, which shows that there is no statistically significant difference between MBFL with IETCR strategy and the original MBFL. INDEX TERMS Software fault localization, Mutation based fault localization, Information entropy, Test case reduction Recent studies [11], [12] have shown that mutation-based fault localization (MBFL) techniques outperform state-ofthe-art SBFL techniques in terms of the fault localization

show abstract

Section: Threats To Validitymentioning

confidence: 99%

Section: B Mutation-based Fault Localizationmentioning

confidence: 99%

Section: B Subject Programsmentioning

confidence: 99%

Section: Threats To Validitymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

Delta4Ms: Improving mutation‐based fault localization by eliminating mutant bias

Liu,

Li,

Han

et al. 2024

Software Testing Verif & Rel

View full text Add to dashboard Cite

Fault localization is a complex, costly and time‐consuming task in software debugging. Numerous automated techniques have been developed to expedite this process. Mutation‐based fault localization (MBFL) is one of the most widely studied techniques which uses mutation analysis to generate mutants for revealing potential faults in the program. However, our theoretical analysis exposes an inherent conflict between the fundamental assumption and the essential meaning of existing MBFL suspiciousness. This conflict is caused by mutant bias. Intuitively, the suspiciousness can be corrected by eliminating the mutant bias for more accurately measuring the faulty probability of the corresponding mutant statement. In this paper, we introduce Delta4Ms, a fault localization approach designed to eliminate mutant bias. Delta4Ms integrates the principles of signal theory, modelling the actual suspiciousness and mutant bias as the desired and false signal components, respectively. Based on theoretical derivation, the average suspiciousness of mutants serves as an estimate of mutant bias. Delta4Ms effectively mitigates mutant bias, extracting the desired signal and yielding corrected suspiciousness for fault localization. To precisely estimate mutant bias, higher order mutants (HOMs) are incorporated. We conduct an extensive experimental evaluation of Delta4Ms on 320 real‐fault programs from Codeflaws. The results indicate that our model significantly outperforms existing SBFL and MBFL techniques, showing a considerable improvement in fault localization effectiveness. We further assessed the robustness of Delta4Ms by examining different HOM ratios and HOM generation strategies. Moreover, Delta4Ms achieves a substantial reduction in mutation execution cost and minimal accuracy loss through the implementation of test case reduction. Finally, we perform preliminary experiments on 15 real‐fault programs from the Defects4J benchmark to assess the generalization of the model's fault localization effectiveness.

show abstract

Test Oracle Prediction for Mutation Based Fault Localization

Zheng

Wang

et al. 2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

Computing reliability: On the differences between software testing and software fault injection techniques

Cited by 21 publications

References 23 publications

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

Delta4Ms: Improving mutation‐based fault localization by eliminating mutant bias

Test Oracle Prediction for Mutation Based Fault Localization

Contact Info

Product

Resources

About