Students Struggle to Explain Their Own Program Code

Abstract

We asked students to explain the structure and execution of their small programs after they had submitted them to a programming exercise. These questions about learner's code (QLCs) were delivered at three occasions in an online and open course in introductory programming as a part of the digital learning material. We make inductive content analysis to research the open-ended text answers we collected. One third of the students struggled to explain their own program code. This estimates possible occurrences of fragile learning at the moment when a student seemingly succeeds in a program writing exercise. Furthermore, we examine correlations between the correctness of the answers with other learning data.Our results indicate that answering properly aligned QLCs correctly has stronger correlation with student success and retention than merely submitting a correct program. Additionally, we present observations on learning event-driven programming to explore QLCs' potential in identifying students' thinking process. CCS CONCEPTS• Social and professional topics → Computing education; • Applied computing → Education; E-learning; Computer-assisted instruction; • Software and its engineering → Publish-subscribe / event-based architectures.

Automated Questions About Learners' Own Code Help to Detect Fragile Prerequisite Knowledge

“…We hypothesized that QLCs could reveal weaknesses in prerequisites which again can lead to low grades or even drop-outs. We found that success rates for the different types of QLCs were comparable to those reported in previous studies [7,8,16] even though the students had passed a CS1 in Python. Furthermore, students answering incorrectly about their own program logic had a lower median for course points compared with those that answered correctly.…”

“…While novice programmers might be able to produce a working solution to a programming problem, some of them might struggle and answer incorrectly even simple questions about their own (functionally correct) program code [8]. Previous research suggests that this phenomena may be due to fragile programming skills in general [13], misconceptions on programming constructs [11,20], inability to trace code [1,10] or that the failing students did not construct the program by themselves [22].…”

“…We partially replicate previous studies on automatically generated Questions about Learners' Code (QLCs) [7,8,16] by adapting selected research questions to new context. The previous studies focused on students' program comprehension during the first weeks of their first programming course (CS1).…”

Jask

“…In previous work we have conducted a small scale experiment where QLCs were used as learning activity in an introductory course on web programming [10]. However, QLCs were prepared manually and accepted open-ended text answers (e.g., "Describe the responsibilities of your outer loop in few words.").…”

Let’s Ask Students About Their Programs, Automatically

“…Early work along these lines is underway. Moreover, we are conducting a preliminary investigation where we pose manually created QLCs to students after a programming assignment [46] (loosely similarly to our main scenario for automatic QLCs). In this pilot study, a system presents students with self-explanation questions that a researcher has tailored for the particular programming assignment and that are thus not chosen automatically or filled in based on facts collected from the student's program.…”