Automated clustering and program repair for introductory programming assignments

Abstract

Providing feedback on programming assignments is a tedious task for the instructor, and even impossible in large Massive Open Online Courses with thousands of students. Previous research has suggested that program repair techniques can be used to generate feedback in programming education. In this paper, we present a novel fully automated program repair algorithm for introductory programming assignments. The key idea of the technique, which enables automation and scalability, is to use the existing correct student solutions to repair the incorrect attempts. We evaluate the approach in two experiments: (I) We evaluate the number, size and quality of the generated repairs on 4,293 incorrect student attempts from an existing MOOC. We find that our approach can repair 97% of student attempts, while 81% of those are small repairs of good quality. (II) We conduct a preliminary user study on performance and repair usefulness in an interactive teaching setting. We obtain promising initial results (the average usefulness grade 3.4 on a scale from 1 to 5), and conclude that our approach can be used in an interactive setting. CCS Concepts • Applied computing → Computerassisted instruction; • Software and its engineering → Software testing and debugging;

Investigating the Essential of Meaningful Automated Formative Feedback for Programming Assignments

“…Massive Open Online Programming Courses provided large datasets of programming assignments, which is critical to make such approaches possible. The efforts 978-1-7281-0810-0/19/$31.00 ©2019 IEEE typically aim at providing student suggestions on repairing their program through measuring the distance between their program and the most similar working program [19]- [21]. Such efforts are still in their early stage for two reasons.…”

TipsC: Tips and Corrections for programming MOOCs

“…Another common approach to feedback generation is clustering where students submissions having similar features are grouped together in clusters. The clusters are created either by using a fixed set of rules or by using machine learning techniques [16,18,5] or by using techniques based on program analysis [6,7,11,4,10]. The feedback is typically generated manually for a representative program in each cluster, and it is customized to other members of the cluster automatically.…”

Targeted Example Generation for Compilation Errors

“…These assignments were completed on a custom web-browser based IDE, which records all intermediate code attempts. This dataset has previously been used by multiple state-of-the-art repair tools in literature [19], [12], [8], [11].…”

“…Potential hardness (x-axis) of resolving an error-group plotted against potential performance improvement (y-axis) offered by our example generation tool. E7 expected ";" in "for" statement Example Feedback 3 for(i=0; i<n; i++) 3 for(i=1; i<=n; i++) (b) EG19 = {E7} Sample programs and top-2 example suggestions, for (a) EG 25 where students with access to TEGCER perform poorly, (b) EG19 where examples seems to work better.…”