Is This a Bug or an Obsolete Test?

Hao, Dan; Lan, Tian; Zhang, Hongyu; Guo, Chuanbin; Zhang, Lu

doi:10.1007/978-3-642-39038-8_25

Cited by 22 publications

(19 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, only the work presented by Hao et al [16] and Herzig and Nagappan [15] classify test failures related to test and infrastructure issues rather than code defects.…”

Section: Related Workmentioning

confidence: 99%

Empirically Detecting False Test Alarms Using Association Rules

Herzig

Nagappan

2015

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering

View full text Add to dashboard Cite

Applying code changes to software systems and testing these code changes can be a complex task that involves many different types of software testing strategies, e.g. system and integration tests. However, not all test failures reported during code integration are hinting towards code defects. Testing large systems such as the Microsoft Windows operating system requires complex test infrastructures, which may lead to test failures caused by faulty tests and test infrastructure issues. Such false test alarms are particular annoying as they raise engineer attention and require manual inspection without providing any benefit. The goal of this work is to use empirical data to minimize the number of false test alarms reported during system and integration testing. To achieve this goal, we use association rule learning to identify patterns among failing test steps that are typically for false test alarms and can be used to automatically classify them. A successful classification of false test alarms is particularly valuable for product teams as manual test failure inspection is an expensive and time-consuming process that not only costs engineering time and money but also slows down product development. We evaluating our approach on system and integration tests executed during Windows 8.1 and Microsoft Dynamics AX development. Performing more than 10,000 classifications for each product, our model shows a mean precision between 0.85 and 0.90 predicting between 34% and 48% of all false test alarms.

show abstract

Section: Related Workmentioning

confidence: 99%

Empirically Detecting False Test Alarms Using Association Rules

Herzig

Nagappan

2015

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering

View full text Add to dashboard Cite

show abstract

“…Besides test case generation, mutation testing, especially mutants, are widely used to simulate faults in the evaluation of software testing techniques [25], [13], [41], [33]. Mutants can be used to simulate faults because previous work [3] presented an empirical evidence that mutants simulate real faults better than manually seeded faults.…”

Section: B Applications Of Mutation Testingmentioning

confidence: 99%

An Empirical Study on the Scalability of Selective Mutation Testing

Zhang

Zhu

Hao

et al. 2014

2014 IEEE 25th International Symposium on Software Reliability Engineering

Self Cite

View full text Add to dashboard Cite

Software testing plays an important role in ensuring software quality by running a program with test suites. Mutation testing is designed to evaluate whether a test suite is adequate in detecting faults. Due to the expensive cost of mutation testing, selective mutation testing was proposed to select a subset of mutants whose effectiveness is similar to the whole set of generated mutants. Although selective mutation testing has been widely investigated in recent years, many people still doubt whether it can suit well for large programs. To study the scalability of selective mutation testing, we systematically explore how the program size impacts selective mutation testing through four projects (including 12 versions all together). Based on the empirical study, for programs smaller than 16 KLOC, selective mutation testing has surprisingly good scalability. In particular, for a program whose number of lines of executable code is E, the number of mutants used in selective mutation testing is proportional to E c , where c is a constant whose value is between 0.05 and 0.25.

show abstract

“…Kim et al [35] further studied the impact of noise in the training data on the effectiveness of defect prediction approaches. Hao et al [3] proposed an approach to predict whether a runtime error is caused by a defect in the source code or the out-of-date test code. Compared to defect prediction approaches, our approach targets a different problem.…”

Section: Defect Predictionmentioning

confidence: 99%

“…The four subjects include two largescale software projects from Microsoft, and two popular open-source software projects from SourceForge 3 . When applying our approach, we considered two possible usage scenarios: the conservative scenario for cautious developers who want to perform cloning operations only when they are sure that the cloning operations would be unlikely to cause consistencymaintenance requirement, and the aggressive scenario for radical developers, who want to avoid a significant proportion of consistency-maintenance-required cloning operations while still being able to perform most intended cloning operations.…”

Section: Introductionmentioning

confidence: 99%

“…When applying our approach, we considered two possible usage scenarios: the conservative scenario for cautious developers who want to perform cloning operations only when they are sure that the cloning operations would be unlikely to cause consistencymaintenance requirement, and the aggressive scenario for radical developers, who want to avoid a significant proportion of consistency-maintenance-required cloning operations while still being able to perform most intended cloning operations. For the first scenario, our approach recommends developers to perform only cloning operations predicted to be very likely to be consistency-maintenance-free; while for the second scenario, out approach warns developers only on cloning operations predicted to be very likely 3. http://sourceforge.net/ to be consistency-maintenance-required.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Predicting Consistency-Maintenance Requirement of Code Clonesat Copy-and-Paste Time

Wang¹,

Dang

Zhang

et al. 2014

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

Abstract-Code clones have always been a double edged sword in software development. On one hand, it is a very convenient way to reuse existing code, and to save coding effort. On the other hand, since developers may need to ensure consistency among cloned code segments, code clones can lead to extra maintenance effort and even bugs. Recently studies on the evolution of code clones show that only some of the code clones experience consistent changes during their evolution history. Therefore, if we can accurately predict whether a code clone will experience consistent changes, we will be able to provide useful recommendations to developers on leveraging the convenience of some code cloning operations, while avoiding other code cloning operations to reduce future consistency maintenance effort. In this paper, we define a code cloning operation as consistency-maintenance-required if its generated code clones experience consistent changes in the software evolution history, and we propose a novel approach that automatically predicts whether a code cloning operation requires consistency maintenance at the time point of performing copy-and-paste operations. Our insight is that whether a code cloning operation requires consistency maintenance may relate to the characteristics of the code to be cloned and the characteristics of its context. Based on a number of attributes extracted from the cloned code and the context of the code cloning operation, we use Bayesian Networks, a machine-learning technique, to predict whether an intended code cloning operation requires consistency maintenance. We evaluated our approach on four subjects -two large-scale Microsoft software projects, and two popular open-source software projects -under two usage scenarios: 1) recommend developers to perform only the cloning operations predicted to be very likely to be consistency-maintenance-free, and 2) recommend developers to perform all cloning operations unless they are predicted very likely to be consistency-maintenance-required. In the first scenario, our approach is able to recommend developers to perform more than 50% cloning operations with a precision of at least 94% in the four subjects. In the second scenario, our approach is able to avoid 37% to 72% consistency-maintenance-required code clones by warning developers on only 13% to 40% code clones, in the four subjects.

show abstract

Is This a Bug or an Obsolete Test?

Cited by 22 publications

References 68 publications

Empirically Detecting False Test Alarms Using Association Rules

Empirically Detecting False Test Alarms Using Association Rules

An Empirical Study on the Scalability of Selective Mutation Testing

Predicting Consistency-Maintenance Requirement of Code Clonesat Copy-and-Paste Time

Contact Info

Product

Resources

About