Let’s Talk <i>With</i> Developers, Not <i>About</i> Developers: A Review of Automatic Program Repair Research

Winter, Emily; Nowack, Vesna; Bowes, David; Counsell, Steve; Hall, Tracy; Haraldsson, Sæmundur Ó.; Woodward, John R.

doi:10.1109/tse.2022.3152089

Cited by 14 publications

(5 citation statements)

References 67 publications

(110 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This fact encourages human studies with APR tools [7,13,45,53] to understand how effective actually the patches in assisting development process. However, a study by Winter et al [49] found that this kind of human studies is still rare, and make only 7% in the Living Review [31], even less for patch adoption experiments.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

On the acceptance by code reviewers of candidate security patches suggested by Automated Program Repair tools

Papotti¹,

Paramitha²,

Massacci³

2022

Preprint

View full text Add to dashboard Cite

Background: Testing and validation of the semantic correctness of patches provided by tools for Automated Program Repairs (APR) has received a lot of attention. Yet, the eventual acceptance or rejection of suggested patches for real world projects by humans patch reviewers has received a limited attention. Objective: To address this issue, we plan to investigate whether (possibly incorrect) security patches suggested by APR tools are recognized by human reviewers. We also want to investigate whether knowing that a patch was produced by an allegedly specialized tool does change the decision of human reviewers. Method: In the first phase, using a balanced design, we propose to human reviewers a combination of patches proposed by APR tools for different vulnerabilities and ask reviewers to adopt or reject the proposed patches. In the second phase, we tell participants that some of the proposed patches were generated by security specialized tools (even if the tool was actually a 'normal' APR tool) and measure whether the human reviewers would change their decision to adopt or reject a patch. Limitations: The experiment will be conducted in an academic setting, and to maintain power, it will focus on a limited sample of popular APR tools and popular vulnerability types.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Key Novelty: Building on these studies [7,13,22,45,49,53], we design an experiment with humans code reviewers to determine if Automated Program Repairs tools effectively support the identification of correct vulnerability fixes.…”

Section: Related Workmentioning

confidence: 99%

On the acceptance by code reviewers of candidate security patches suggested by Automated Program Repair tools

Papotti¹,

Paramitha²,

Massacci³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In our prior work [4], we conducted a literature review to assess the extent to which human factors are considered within the APR literature, as well as evaluating the quality of existing human studies. We found that there were very few human studies currently in APR -just 7% of the 260 papers we reviewed.…”

Section: Introductionmentioning

confidence: 99%

How do Developers Really Feel About Bug Fixing? Directions for Automatic Program Repair

Winter

Bowes

Counsell

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Automatic program repair (APR) is a rapidly advancing field of software engineering that aims to supplement or replace manual bug fixing with an automated tool. For APR to be successfully adopted in industry, it is vital that APR tools respond to developer needs and preferences. However, very little research has considered developers' general attitudes to APR or developers' current bug fixing practices (the activity APR aims to replace). This paper responds to this gap by reporting on a survey of 386 software developers about their bug finding and fixing practices and experiences, and their instinctive attitudes towards APR. We find that bug finding and fixing is not necessarily as onerous for developers as has often been suggested, being rated as more satisfying than developers' general work. The fact that developers derive satisfaction and benefit from bug fixing indicates that APR adoption is not as simple as APR replacing an unwanted activity. When it comes to potential APR approaches, we find a strong preference for developers being kept in the loop (for example, choosing between different fixes or validating fixes) as opposed to a fully automated process. This suggests that advances in APR should be careful to consider the agency of the developer, as well as what information is presented to developers alongside fixes. It also indicates that there are key barriers related to trust that would need to be overcome for full scale APR adoption, supported by the fact that even those developers who stated that they were positive about APR listed several caveats and concerns. We find very few statistically significant relationships between particular demographic variables (for example, developer experience, age, education) and key attitudinal variables, suggesting that developers' instinctive attitudes towards APR are little influenced by experience level but are held widely across the developer community.

show abstract

“…However, it is resource-consuming and costprohibitive (Weiß et al, 2007;Planning, 2002;Jørgensen and Shepperd, 2007). A reliable automatic program repair (APR) system is thus crucial to reduce manual debugging efforts and development time (Gazzola et al, 2019;Winter et al, 2023).…”

Section: Introductionmentioning

confidence: 99%

Towards Low-Resource Automatic Program Repair with Meta-Learning and Pretrained Language Models

Wang,

Hoi

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Automatic program repair (APR) has gained increasing attention as an essential technique in software development to reduce manual debugging efforts and boost developers' productivity. Recent advances in deep learning (DL) based models have demonstrated promising results by learning from large-scale bug-fix examples in a data-driven manner. However, in practical scenarios, software bugs have an imbalanced distribution, and the fixing knowledge learned by APR models often only capture the patterns of frequent error types, making it inapplicable to handle the rare error types. To address this limitation, we investigate a novel task of low-resource APR, and propose Meta-APR, a new meta-learning framework integrated with code pretrained language models to generate fixes for low-resource bugs with limited training samples. Our Meta-APR learns better errorspecific knowledge from high-resource bugs through efficient first-order meta-learning optimization, which allows for a faster adaptation to the target low-resource bugs. Besides, while we adopt CodeT5, a pretrained code-aware encoder-decoder Transformer, as the backbone model for Meta-APR, it is a model-agnostic framework that can be integrated with any neural models. Extensive experimental results on three benchmarks in various programming languages verify the superiority of our method over existing DL-based APR approaches.

show abstract

Let’s Talk With Developers, Not About Developers: A Review of Automatic Program Repair Research

Cited by 14 publications

References 67 publications

On the acceptance by code reviewers of candidate security patches suggested by Automated Program Repair tools

On the acceptance by code reviewers of candidate security patches suggested by Automated Program Repair tools

How do Developers Really Feel About Bug Fixing? Directions for Automatic Program Repair

Towards Low-Resource Automatic Program Repair with Meta-Learning and Pretrained Language Models

Contact Info

Product

Resources

About