Spectrum-based fault localization in software product lines

Arrieta, Aitor; Segura, Sergio; Markiegi, Urtzi; Sagardui, Goiuria; Etxeberria, Leire

doi:10.1016/j.infsof.2018.03.008

Cited by 40 publications

(28 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the cases containing a single incorrect statement (single-bug), our results show that VARCOP significantly outperformed S-SBFL, SBFL, and Arrieta et al [10] in all 30/30 metrics by 33%, 50%, and 95% in Rank, respectively. Impressively, VARCOP correctly ranked the bugs at the top-3 positions in +65% of the cases.…”

Section: Introductionmentioning

confidence: 72%

“…How accurate is VARCOP in localizing variability bugs? And how is it compared to the state-of-the-art approaches [9], [10], [14]? RQ2: Intrinsic Analysis.…”

Section: Empirical Methodologymentioning

confidence: 99%

“…Consequently, their resulting suspicious spaces are often large. Meanwhile, by Arrieta et al [10], SBFL is adapted to localize bugs at the feature-level. Particularly, each sampled product is considered as a test (i.e., passed tests are passing products, and failed tests are failing products), and the spectra record the feature selections in each product.…”

Section: The Root Cause Of Variability Failuresmentioning

confidence: 99%

“…We conducted experiments to evaluate VARCOP in both single-bug and multiple-bug settings on a dataset of 1,570 versions (cases) containing variability bug(s) [18]. We compared VARCOP with the state-of-the-art approaches including (SBFL) [11], [14]- [17], the combination of the slicing method and SBFL (S-SBFL) [21], [22], and Arrieta et al [10] using 30 most popular SBFL ranking metrics [11], [14], [15].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Variability Fault Localization Approach for Software Product Lines

Nguyen,

Ngo,

Nguyen

et al. 2021

Preprint

View full text Add to dashboard Cite

Software fault localization is one of the most expensive, tedious, and time-consuming activities in program debugging. This activity becomes even much more challenging in Software Product Line (SPL) systems due to variability of failures. These unexpected behaviors are induced by variability faults which can only be exposed under some combinations of system features. The interaction among these features causes the failures of the system. Although localizing bugs in single-system engineering has been studied in-depth, variability fault localization in SPL systems still remains mostly unexplored. In this article, we present VARCOP, a novel and effective variability fault localization approach. For an SPL system failed by variability bugs, VARCOP isolates suspicious code statements by analyzing the overall test results of the sampled products and their source code. The isolated suspicious statements are the statements related to the interaction among the features which are necessary for the visibility of the bugs in the system. In VARCOP, the suspiciousness of each isolated statement is assessed based on both the overall test results of the products containing the statement as well as the detailed results of the test cases executed by the statement in these products. On a large public dataset of buggy SPL systems, our empirical evaluation shows that VARCOP significantly improves two state-of-the-art techniques by 33% and 50% in ranking the incorrect statements in the systems containing a single bug each. In about two-thirds of the cases, VARCOP correctly ranks the buggy statements at the top-3 positions in the ranked lists. For the cases containing multiple bugs, VARCOP outperforms the state-of-the-art approaches 2 times and 10 times in the proportion of bugs localized at the top-1 positions. Especially, in 22% and 65% of the buggy versions, VARCOP correctly ranks at least one bug in a system at the top-1 and top-5 positions.

show abstract

Section: Introductionmentioning

confidence: 72%

“…How accurate is VARCOP in localizing variability bugs? And how is it compared to the state-of-the-art approaches [9], [10], [14]? RQ2: Intrinsic Analysis.…”

Section: Empirical Methodologymentioning

confidence: 99%

Section: The Root Cause Of Variability Failuresmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Variability Fault Localization Approach for Software Product Lines

Nguyen,

Ngo,

Nguyen

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Such information will be used for calculating the probability that program entities (such as statements) incur a fault and generating a ranking list of them for developers to inspect code elements. SBFL is simple to implement and previous studies have shown that SBFL is promising to reduce the cost of software debugging [8], [9]. However, when applied to large-scale programs, SBFL still has the problem of poor fault localization accuracy.…”

Section: Introductionmentioning

confidence: 99%

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Mutation-based fault localization (MBFL) is a recently proposed technique with the advantage of high fault localization accuracy. However, such a mutation analysis based technique is difficult to be accepted by industry due to its huge computational cost on mutation analysis. There are three ways to improve MBFL's efficiency, which are reducing the number of mutants, optimizing the mutants' execution process, and reducing the number of test cases. The former two ways have been mainly studied and shown promising results, but for the latter way, the related studies are limited since this kind of method will reduce the precision of MBFL. In this paper, we mainly focus on the latter way and propose an information entropy based test case reduction (IETCR) strategy for MBFL. In particular, we first calculate the entropy change of test cases and select a proportion of them according to their value. Then we use a reduced test suite to execute mutants. To show the effectiveness of the IETCR strategy, we choose six real-world programs with faulty versions. In terms of mutation reduction rate, we find MBFL with the IETCR strategy can reduce 56.3%∼88.3% cost while keeping almost the same fault localization accuracy when compared to the original MBFL without test case reduction. Moreover, we use Wilcoxon signed-rank test for statistical analysis, which shows that there is no statistically significant difference between MBFL with IETCR strategy and the original MBFL. INDEX TERMS Software fault localization, Mutation based fault localization, Information entropy, Test case reduction Recent studies [11], [12] have shown that mutation-based fault localization (MBFL) techniques outperform state-ofthe-art SBFL techniques in terms of the fault localization

show abstract

CRMF: A fault localization approach based on class reduction and method call frequency

Liu

Wang

et al. 2022

Softw Pract Exp

View full text Add to dashboard Cite

Identifying the location of faults in real-world programs is one of the costly processes during software debugging. To reduce the debugging effort, various fault localization techniques have been proposed in recent years. Spectrum-based fault localization (SBFL) is one kind of widely investigated fault localization technique. Most SBFL techniques first calculate the suspiciousness of programelements (such as statements, methods) to be faulty using the coverage information and execution results of tests. Then a rank list of program elements is generated according to their suspiciousness. However, some SBFL techniques only consider the binary coverage information (i.e., whether the program element is covered) but ignore some of the tests' running behaviors, such as the execution frequency when faults occur in the iteration entities or loop bodies, which are more likely to be faulty followed the propagation-infection-execution model. The execution frequency based techniques only replace the feature items of the existing formula limiting their effectiveness in fault localization.In this article, we propose a fault localization technique, class reduction and method call frequency (CRMF), which utilizes mutation analysis and information retrieval techniques. In particular, CRMF first uses mutation analysis to identify and reduce the classes, in which the program elements with a low probability of being faulty. Then we propose a new suspiciousness formula that applies information retrieval and considers method call frequency. To evaluate the effectiveness of CRMF, we conduct empirical studies on 264 real-world programs from the Defects4J benchmark. Final results show that CRMF outperforms the statement frequency based technique FLSF and SBFL techniques (i.e., Ochiai, OP2, Tarantula, and Dstar) in both single-fault programs and multiple-fault programs. Specifically, CRMF can rank 29, 74, and 112 faults at the top 1, 3, 5 ranks and achieve a higher mean reciprocal rank for single-fault programs and multiple-fault programs. Finally, we discuss the essence of CRMF and analyze its effectiveness on multi-fault programs in detail.

show abstract

Spectrum-based fault localization in software product lines

Cited by 40 publications

References 60 publications

A Variability Fault Localization Approach for Software Product Lines

A Variability Fault Localization Approach for Software Product Lines

IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization

CRMF: A fault localization approach based on class reduction and method call frequency

Contact Info

Product

Resources

About