Does the Failing Test Execute a Single or Multiple Faults? An Approach to Classifying Failing Tests

Yu, Zhongxing; Bai, Chenggang; Cai, Kai-Yuan

doi:10.1109/icse.2015.102

Cited by 19 publications

(13 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hajri et al [82] used a logistical progression model to automatically classify the test cases for use case-driven testing in product lines. Yu et al [318] proposed a technique that can be used to distinguish failing tests that executed a single fault from those that executed multiple faults. The technique combines information from a set of fault localization ranked lists, each produced for a certain failing test, and the distance between a failing test and the passing test that most resembles it.…”

Section: Test Casementioning

confidence: 99%

Predictive Models in Software Engineering: Challenges and Opportunities

Yang

Xia

et al. 2022

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Predictive models are one of the most important techniques that are widely applied in many areas of software engineering. There have been a large number of primary studies that apply predictive models and that present well-performed studies in various research domains, including software requirements, software design and development, testing and debugging and software maintenance. This paper is a first attempt to systematically organize knowledge in this area by surveying a body of 421 papers on predictive models published between 2009 and 2020. We describe the key models and approaches used, classify the different models, summarize the range of key application areas, and analyze research results. Based on our findings, we also propose a set of current challenges that still need to be addressed in future work and provide a proposed research road map for these opportunities.

show abstract

Section: Test Casementioning

confidence: 99%

Predictive Models in Software Engineering: Challenges and Opportunities

Yang

Xia

et al. 2022

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…However, only further studies with more diverse programs and faults can strengthen claims for generalisation. Additionally, the reported results are strictly based on the failure-to-single-fault assumption [11], and may not generalise for multiple-fault failures [60]. We will consider overlapping clustering [51,52,56,58] to handle such cases in the future.…”

Section: Threats To Validitymentioning

confidence: 99%

Improving Test Distance for Failure Clustering with Hypergraph Modelling

An,

Yoon,

Whang

et al. 2021

Preprint

View full text Add to dashboard Cite

Automated debugging techniques, such as Fault Localisation (FL) or Automated Program Repair (APR), are typically designed under the Single Fault Assumption (SFA). However, in practice, an unknown number of faults can independently cause multiple test case failures, making it difficult to allocate resources for debugging and to use automated debugging techniques. Clustering algorithms have been applied to group the test failures according to their root causes, but their accuracy can often be lacking due to the inherent limits in the distance metrics for test cases. We introduce a new test distance metric based on hypergraphs and evaluate their accuracy using multi-fault benchmarks that we have built on top of Defects4J and SIR. Results show that our technique, Hybiscus, can automatically achieve perfect clustering (i.e., the same number of clusters as the ground truth number of root causes, with all failing tests with the same root cause grouped together) for 418 out of 605 test runs with multiple test failures. Better failure clustering also allows us to separate different root causes and apply FL techniques under SFA, resulting in saving up to 82% of the total wasted effort when compared to the state-of-the-art technique for multiple fault localisation.

show abstract

“…Therefore, the findings show that Expense score and Exam score are the most utilized by the selected papers, respectively. [17,27,29,33,[52][53][54][55][56][57] Exam score [43, 49-51, 63, 64, 67] Wasted effort [46][47][48]65] Precision & recall [44,45,66] e) Fault Isolation Fault isolation is the process of isolating faults caused by different failures into separate clusters for efficient and more effective multiple fault localization. Most of the selected papers that utilized method such as parallel debugging used various clustering algorithms to isolate faults.…”

Section: D) the Evaluation Metrics Utilized Across The Selected Studiesmentioning

confidence: 99%

Spectrum-based Fault Localization Techniques Application on Multiple-Fault Programs: A Review

Zakari¹,

Abdullahi²,

Shagari³

et al. 2020

GJCST

View full text Add to dashboard Cite

Software fault localization is one of the most tedious and costly activities in program debugging in the endeavor to identify faults locations in a software program. In this paper, the studies that used spectrum-based fault localization (SBFL) techniques that makes use of different multiple fault localization debugging methods such as one-bug-at-a-time (OBA) debugging, parallel debugging, and simultaneous debugging in localizing multiple faults are classified and critically analyzed in order to extensively discuss the current research trends, issues, and challenges in this field of study. The outcome strongly shows that there is a high utilization of OBA debugging method, poor fault isolation accuracy, and dominant use of artificial faults that limit the existing techniques applicability in the software industry.

show abstract

Does the Failing Test Execute a Single or Multiple Faults? An Approach to Classifying Failing Tests

Cited by 19 publications

References 44 publications

Predictive Models in Software Engineering: Challenges and Opportunities

Predictive Models in Software Engineering: Challenges and Opportunities

Improving Test Distance for Failure Clustering with Hypergraph Modelling

Spectrum-based Fault Localization Techniques Application on Multiple-Fault Programs: A Review

Contact Info

Product

Resources

About