Learning From Errors: Exploring the Effectiveness of Enhanced Error Messages in Learning to Program

Abstract

Error messages provided by the programming environments are often cryptic and confusing to learners. This study explored the effectiveness of enhanced programming error messages (EPEMs) in a Python-based introductory programming course. Participants were two groups of middle school students. The control group only received raw programming error messages (RPEMs) and had 35 students. The treatment group received EPEMs and had 33 students. During the class, students used an automated assessment tool called Mulberry to practice their programming skill. Mulberry automatically collected all the solutions students submitted when solving programming problems. Data analysis was based on 6339 student solutions collected by Mulberry. Our results showed that EPEMs did not help to reduce student errors or improve students’ performance in debugging. The ineffectiveness of EPEMs may result from reasons such as the inaccuracy of the interpreter’s error messages or students not reading the EPEMs. However, the viewpoint of productive failure may provide a better explanation of the ineffectiveness of EPEMs. The failures in coding and difficulties in debugging can be resources for learning. We recommend that researchers reconsider the role of errors in code and investigate whether and how failures and debugging contribute to the learning of programming.

Common Errors in High School Novice Programming

“…The study suggests that students' competency in improving their code is important to their success in introductory programming. Also, in [19] authors explored the effectiveness of Enhanced Programming Error Messages (EPEMs) in a Python-based introductory programming course in the middle school using an automated assessment tool Mulberry.…”

Untitled

“…Hattori and Kameda [32] and Becker et al [40,41] have developed tools for enhancing Java compiler error messages providing more information and changing description text for some common errors (through special front-end for the compiler), achieving good results. On the other hand, Zhou, et al [42] have not found the enhanced messages to be useful for help to reduce student errors or improve students' performance in debugging. Becker et al [43] present reliable evidence that this heuristic technique "fix the first error and ignore the rest" is trustworthy.…”

“…There is no need of modify compiler main program. Other challenging ideas include modify the compiler code for dealing with cascade error messages like presented by [40,41] and [32]; and, of course, enhancing error messages like done by [38,39,40,41,42].…”

Formative Assessment as a Learning Method for Introductory Programming

“…However, it can help for recalling the programming concept and motivating their engagement. Learning from errors is another effective approach of making students learn in programming [4]. It helps students to understand the frequent errors they make while coding and the error messages from the compiler.…”