A human study of fault localization accuracy

Fry, Zachary P.; Weimer, Westley

doi:10.1109/icsm.2010.5609691

Cited by 41 publications

(24 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Amazon.com's Mechanical Turk crowdsourcing service deserves a detailed mention; it is effective as a means of obtaining a diverse population, but requires special consideration to ensure the overall quality of the data set. Previous work has shown that the Mechanical Turk participants can be effective when large populations are required [17,33] -including for software engineering studies [11]. However, when offered a reward for an anonymous service, people may attempt to receive compensation without giving their best effort.…”

Section: Participant Selectionmentioning

confidence: 99%

See 1 more Smart Citation

A human study of patch maintainability

Fry

Landau

Weimer

2012

Proceedings of the 2012 International Symposium on Software Testing and Analysis

Self Cite

View full text Add to dashboard Cite

Identifying and fixing defects is a crucial and expensive part of the software lifecycle. Measuring the quality of bug-fixing patches is a difficult task that affects both functional correctness and the future maintainability of the code base. Recent research interest in automatic patch generation makes a systematic understanding of patch maintainability and understandability even more critical.We present a human study involving over 150 participants, 32 real-world defects, and 40 distinct patches. In the study, humans perform tasks that demonstrate their understanding of the control flow, state, and maintainability aspects of code patches. As a baseline we use both human-written patches that were later reverted and also patches that have stood the test of time to ground our results. To address any potential lack of readability with machine-generated patches, we propose a system wherein such patches are augmented with synthesized, human-readable documentation that summarizes their effects and context. Our results show that machine-generated patches are slightly less maintainable than human-written ones, but that trend reverses when machine patches are augmented with our synthesized documentation. Finally, we examine the relationship between code features (such as the ratio of variable uses to assignments) with participants' abilities to complete the study tasks and thus explain a portion of the broad concept of patch quality.

show abstract

Section: Participant Selectionmentioning

confidence: 99%

“…We mitigate this threat by choosing as features the union of those mentioned by participants and those used in previous studies exploring the maintenance process [8,11].…”

Section: Threats To Validitymentioning

confidence: 99%

A human study of patch maintainability

Fry

Landau

Weimer

2012

Proceedings of the 2012 International Symposium on Software Testing and Analysis

Self Cite

View full text Add to dashboard Cite

show abstract

“…Software engineering research can also benefit from crowdsourcing. It can be used to conduct human studies [59], [136], [215], [242], [243]. We summarised a few studies on using crowd evaluation for software engineering research in Table 7.…”

Section: Crowdsourcing For Other Software Engineering Activitiesmentioning

confidence: 99%

A survey of the use of crowdsourcing in software engineering

Mao

Capra

Harman

et al. 2017

Journal of Systems and Software

294

231

View full text Add to dashboard Cite

Crowdsourcing can be used to support software engineering activities and research into these activities. In this paper we provide a comprehensive survey of the use of crowdsourcing to support software engineering activities (Crowdsourced Software Engineering), seeking to cover all literature on this topic. We describe the software engineering domains, tasks and applications for crowdsourcing and the platforms and stakeholders involved in realising Crowdsourced Software Engineering solutions. We also expose trends, issues and opportunities for Crowdsourced Software Engineering.Please cite as: Ke Mao, Licia Capra, Mark Harman and Yue Jia. A Survey of the Use of Crowdsourcing in SoftwareEngineering.

show abstract

“…Fry et al [9] presented an observational study on the accuracy of human subjects in locating faults. They discovered that certain types of faults are difficult to locate for humans.…”

Section: Related Workmentioning

confidence: 99%

An empirical study about the effectiveness of debugging when random test cases are used

Ceccato¹,

Marchetto²,

Mariani³

et al. 2012

2012 34th International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Abstract-Automatically generated test cases are usually evaluated in terms of their fault revealing or coverage capability. Beside these two aspects, test cases are also the major source of information for fault localization and fixing. The impact of automatically generated test cases on the debugging activity, compared to the use of manually written test cases, has never been studied before.In this paper we report the results obtained from two controlled experiments with human subjects performing debugging tasks using automatically generated or manually written test cases. We investigate whether the features of the former type of test cases, which make them less readable and understandable (e.g., unclear test scenarios, meaningless identifiers), have an impact on accuracy and efficiency of debugging. The empirical study is aimed at investigating whether, despite the lack of readability in automatically generated test cases, subjects can still take advantage of them during debugging.

show abstract

A human study of fault localization accuracy

Cited by 41 publications

References 22 publications

A human study of patch maintainability

A human study of patch maintainability

A survey of the use of crowdsourcing in software engineering

An empirical study about the effectiveness of debugging when random test cases are used

Contact Info

Product

Resources

About