Impact of Steps, Instruction, and Motivation on Learning Symbolic Reasoning Using an Online Tool

Abstract

Several research studies have shown the benefits of code tracing to promote student understanding of program behavior. While code tracing on specific input values is a useful starting point, students ultimately need to be able to reason rigorously and logically about the correctness of their code on all (i.e., arbitrary) inputs. Otherwise, they may make false generalizations and may achieve only a shallow understanding. Results of a multi-semester experiment to answer the following research questions: (1) With or without steps, can students learn the basics of tracing code on symbolic input values using an online tool? And how important is classroom instruction? (2) What is the impact of motivation on student attitudes in learning to reason with such a tool? Data was obtained from 297 subjects who used the online reasoning tool in a second-year software development course for CS majors. Analysis indicates that students can do symbolic reasoning to trace code and that instruction and motivation have significant impact.

Tool-Aided Learning of Code Reasoning with Abstraction in the CS Curriculum

“…A key aim of using automated tools is to understand the specific fine-grain learning difficulties students face when reasoning about code so that appropriate interventions may be developed (Cook et al, 2018;Fowler et al, 2019;Priester et al, 2016). By "fine-grain," we mean understanding student difficulties at a resolution that exceeds identifying high-level constructs that might present challenges (e.g., functions, loops, parameter passing) to reveal the underlying cause(s) of a learning roadblock (e.g., a missing algebraic foundation or a flaw in the student's mental model of variable storage).…”

“…A similar pattern is observed between the overall exam scores, where fall students performed significantly better, with a p value of 0.0005. While this would point directly to the benefit of classroom instruction, student motivation may be a con founding factor and is discussed in (Fowler et al, 2019). Regardless, the important point is that symbolic reasoning can be learned by a majority of students.…”

“…Additional details of symbolic reasoning studies in pinpointing specific difficulties for individual students and subgroups may be found in (Cook et al, 2018). A related study in (Fowler et al, 2019) explores the role of motivation in learning symbolic reasoning.…”