A weighted fuzzy classification approach to identify and manipulate coincidental correct test cases for fault localization

Liu, Yong; Li, Meiying; Wu, Yonghao; Li, Zheng

doi:10.1016/j.jss.2019.01.056

Cited by 23 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our study, we use M T P as the mutation reduction performance metric and EXAM score as the fault localization accuracy metric. The former metric is popularly used in evaluating the cost of MBFL [12], [21], [32], and the latter one is also a commonly used fault localization metric in previous studies [45], [46]. Therefore, the performance metrics used in our study can reflect the real situation in evaluating MFL methods.…”

Section: Threats To Validitymentioning

confidence: 99%

“…2) EXAM Score Metric EXAM score [44] (EXAM ) is the percentage of program elements that have to be inspected until finding the exact faulty element. It is a commonly used metric for fault localization techniques, and a lower EXAM indicates a better fault localization technique [45], [46]. The EXAM measures the relative position of the faulty element in the ranking list, and the formula of EXAM can be defined as follows.…”

Section: ) Mutant-test-pair Metricmentioning

confidence: 99%

See 1 more Smart Citation

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

Wang

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

Mutation-based fault localization (MBFL) is a recently proposed technique with the advantage of high fault localization accuracy. However, such a mutation analysis based technique is difficult to be accepted by industry due to its huge computational cost on mutation analysis. There are three ways to improve MBFL's efficiency, which are reducing the number of mutants, optimizing the mutants' execution process, and reducing the number of test cases. The former two ways have been mainly studied and shown promising results, but for the latter way, the related studies are limited since this kind of method will reduce the precision of MBFL. In this paper, we mainly focus on the latter way and propose an information entropy based test case reduction (IETCR) strategy for MBFL. In particular, we first calculate the entropy change of test cases and select a proportion of them according to their value. Then we use a reduced test suite to execute mutants. To show the effectiveness of the IETCR strategy, we choose six real-world programs with faulty versions. In terms of mutation reduction rate, we find MBFL with the IETCR strategy can reduce 56.3%∼88.3% cost while keeping almost the same fault localization accuracy when compared to the original MBFL without test case reduction. Moreover, we use Wilcoxon signed-rank test for statistical analysis, which shows that there is no statistically significant difference between MBFL with IETCR strategy and the original MBFL. INDEX TERMS Software fault localization, Mutation based fault localization, Information entropy, Test case reduction Recent studies [11], [12] have shown that mutation-based fault localization (MBFL) techniques outperform state-ofthe-art SBFL techniques in terms of the fault localization

show abstract

Section: Threats To Validitymentioning

confidence: 99%

Section: ) Mutant-test-pair Metricmentioning

confidence: 99%

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

Wang

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…We adopt four performance metrics (i.e., EXAM, Top-N, MAP and MTP) to evaluate the fault localization effectiveness of our proposed approach. These metrics are selected based on their extensive use in prior fault localization research [56,57].…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…To alleviate this, we use EXAM, Top-N and MAP to evaluate the effectiveness of fault localization and MTP to evaluate the efficiency. EXAM has been widely used in evaluating the effectiveness of fault localization in previous studies [56,57]. The metric of Top-N measures the absolute ranks of faulty program statements [60].…”

Section: Construct Validitymentioning

confidence: 99%

Delta4Ms: Improving mutation‐based fault localization by eliminating mutant bias

Liu,

Li,

Han

et al. 2024

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

Fault localization is a complex, costly and time‐consuming task in software debugging. Numerous automated techniques have been developed to expedite this process. Mutation‐based fault localization (MBFL) is one of the most widely studied techniques which uses mutation analysis to generate mutants for revealing potential faults in the program. However, our theoretical analysis exposes an inherent conflict between the fundamental assumption and the essential meaning of existing MBFL suspiciousness. This conflict is caused by mutant bias. Intuitively, the suspiciousness can be corrected by eliminating the mutant bias for more accurately measuring the faulty probability of the corresponding mutant statement. In this paper, we introduce Delta4Ms, a fault localization approach designed to eliminate mutant bias. Delta4Ms integrates the principles of signal theory, modelling the actual suspiciousness and mutant bias as the desired and false signal components, respectively. Based on theoretical derivation, the average suspiciousness of mutants serves as an estimate of mutant bias. Delta4Ms effectively mitigates mutant bias, extracting the desired signal and yielding corrected suspiciousness for fault localization. To precisely estimate mutant bias, higher order mutants (HOMs) are incorporated. We conduct an extensive experimental evaluation of Delta4Ms on 320 real‐fault programs from Codeflaws. The results indicate that our model significantly outperforms existing SBFL and MBFL techniques, showing a considerable improvement in fault localization effectiveness. We further assessed the robustness of Delta4Ms by examining different HOM ratios and HOM generation strategies. Moreover, Delta4Ms achieves a substantial reduction in mutation execution cost and minimal accuracy loss through the implementation of test case reduction. Finally, we perform preliminary experiments on 15 real‐fault programs from the Defects4J benchmark to assess the generalization of the model's fault localization effectiveness.

show abstract

“…As such, it is necessary to generate test cases with "better" quality to improve the chance of faulty statements being ranked as highly suspicious. To this end, some researchers have improved the efficiency of SBFL from the perspective of identifying coincidental correctness (CC) test cases [9][10][11][12][13].…”

mentioning

confidence: 99%

An Approach of Improving the Efficiency of Software Fault Localization based on Feedback Ranking Information

Yang,

Ma,

Guo

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Fault localization, a critical process of software debugging, can be implemented by ranking program statements according to their suspiciousness of being faulty, which, in turn, is calculated based on the execution behaviors of test cases. The performance of fault localization will deteriorate if the actual faulty statement is ranked low in suspiciousness. Intuitively speaking, the quality of the used test cases affects the suspiciousness ranking and thus the efficacy of fault localization. As such, it is necessary to generate test cases with “better” quality to improve the chance of faulty statements being ranked as highly suspicious. In this paper, we propose a software fault localization approach based on feedback ranking information, namely FLFR, according to an improved genetic algorithm. The starting point of the new method is the execution of a set of test cases, which gives a preliminary suspiciousness ranking of program statements. The improved genetic algorithm is iteratively applied to generate new test cases. The new method is evaluated through a series of experiments on four C programs and two Java programs. The experimental results show that the test cases automatically generated by the method can improve the suspiciousness ranking of the faulty statement, and thus enhance the performance of fault localization.

show abstract

A weighted fuzzy classification approach to identify and manipulate coincidental correct test cases for fault localization

Cited by 23 publications

References 23 publications

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

Delta4Ms: Improving mutation‐based fault localization by eliminating mutant bias

An Approach of Improving the Efficiency of Software Fault Localization based on Feedback Ranking Information

Contact Info

Product

Resources

About