InFix: Automatically Repairing Novice Program Inputs

Endres, Madeline; Sakkas, Georgios K.; Cosman, Benjamin; Jhala, Ranjit; Weimer, Westley

doi:10.1109/ase.2019.00045

Cited by 12 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We looked at all the papers that emerged from these searches and, by using the protocol described in Section 4 to determine whether a paper included a human study, found one APR human study that was not included in the Living Review. This human study [72] confirms our core findings in that it is a minimally-described experiment (with participants asked to review human-written repairs and automated repairs), featuring undergraduate students and Amazon Mechanical Turk workers as its participants. As a result, we are confident that our quality assessment of human factors within APR is representative of the APR literature.…”

Section: Threats To Validitysupporting

confidence: 79%

Let’s Talk With Developers, Not About Developers: A Review of Automatic Program Repair Research

Winter

Nowack

Bowes

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Automatic program repair (APR) offers significant potential for automating some coding tasks. Using APR could reduce the high costs historically associated with fixing code faults and deliver significant benefits to software engineering. Adopting APR could also have profound implications for software developers' daily activities, transforming their work practices. To realise the benefits of APR it is vital that we consider how developers feel about APR and the impact APR may have on developers' work. Developing APR tools without consideration of the developer is likely to undermine the success of APR deployment. In this paper, we critically review how developers are considered in APR research by analysing how human factors are treated in 260 studies from Monperrus's Living Review of APR. Over half of the 260 studies in our review were motivated by a problem faced by developers (e.g., the difficulty associated with fixing faults). Despite these human-oriented motivations, fewer than 7% of the 260 studies included a human study. We looked in detail at these human studies and found their quality mixed (for example, one human study was based on input from only one developer). Our results suggest that software developers are often talked about in APR studies, but are rarely talked with. A more comprehensive and reliable understanding of developer human factors in relation to APR is needed. Without this understanding, it will be difficult to develop APR tools and techniques which integrate effectively into developers' workflows. We recommend a future research agenda to advance the study of human factors in APR.

show abstract

Section: Threats To Validitysupporting

confidence: 79%

Let’s Talk With Developers, Not About Developers: A Review of Automatic Program Repair Research

Winter

Nowack

Bowes

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…Several studies have evaluated specific system features that support program construction. These features include, for example, next-step hints and textual explanations [26,27], hints generated from the inputs entered to programs [12], and refactoring support in a visual programming environment [33]. Learner-generated written annotations have been used for constructing tutorials shown to other learners [14].…”

Section: System Evaluationsmentioning

confidence: 99%

“…Others simply used MTurk workers as an additional population (e.g. [12,34]). Some studies have focused solely on MTurk workers.…”

Section: Study Designs and Data Qualitymentioning

confidence: 99%

Crowdsourcing in Computing Education Research: Case Amazon MTurk

Hellas

Zavgorodniaia

Sorva

2020

Koli Calling '20: Proceedings of the 20th Koli Calling International Conference on Computing Education Research

View full text Add to dashboard Cite

Crowdsourcing platforms such as Amazon MTurk provide access to a human workforce that can be given tasks to complete online for a fee. In this article, we review studies in computing education research (CER) that rely on crowdsourcing; we also describe our own experiences of using Amazon MTurk for a CER study. We discuss challenges in recruiting workers with specific backgrounds-such as no programming experience-and considerations in filtering out unreliable research participants. Combining recommendations from the literature with the lessons that we learned whilst conducting our study, we synthesize advice for researchers in CER who are considering crowdsourcing. In our case study, we did not find widespread foul play by crowdsourced workers and, overall, our experiences and the literature suggest that crowdsourced CER is feasible. It is, however, uncertain to what extent crowdsourced data can produce answers that apply to specific educational contexts. More research is needed before definitive conclusions can be drawn about the validity and generalizability of crowdsourced CER. CCS CONCEPTS• Social and professional topics → Computing education.

show abstract