AlloyFL: a fault localization framework for Alloy

Khan, Tanvir Ahmed; Sullivan, Allison; Wang, Kaiyuan

doi:10.1145/3468264.3473116

Cited by 2 publications

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our evaluation uses simple program execution spectra derived from Java-based systems. Spectra derived from other languages [3,32,48,56,58,59] or involving more complex types of elements (e.g., data flow edges [49]) could have different characteristics that impact the results.…”

Section: Threats To Validitymentioning

confidence: 99%

Improving Spectrum-Based Localization of Multiple Faults by Iterative Test Suite Reduction

Callaghan,

Fischer

2023

Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Spectrum-based fault localization (SBFL) works well for single-fault programs but its accuracy decays for increasing fault numbers. We present FLITSR (Fault Localization by Iterative Test Suite Reduction), a novel SBFL extension that improves the localization of a given base metric specifically in the presence of multiple faults. FLITSR iteratively selects reduced versions of the test suite that better localize the individual faults in the system. This allows it to identify and re-rank faults ranked too low by the base metric because they were masked by other program elements.We evaluated FLITSR over method-level spectra from an existing large synthetic dataset comprising 75000 variants of 15 opensource projects with up to 32 injected faults, as well as method-and statement-level spectra from a new dataset with 326 true multi-fault versions from the Defects4J benchmark set containing up to 14 real faults. For all three spectrum types we consistently see substantial reductions of the average wasted efforts at different fault levels, of 30%-90% over the best base metric, and generally similarly large increases in precision and recall, albeit with larger variance across the underlying projects. For the method-level real faults, FLITSR also substantially outperforms GRACE, a state-of-the-art learning-based fault localizer. CCS CONCEPTS• Software and its engineering → Software testing and debugging.

show abstract