Adaptive Scaffolding in Block-Based Programming via Synthesizing New Tasks as Pop Quizzes

Abstract

Block-based programming environments are increasingly used to introduce computing concepts to beginners. However, novice students often struggle in these environments, given the conceptual and open-ended nature of programming tasks. To effectively support a student struggling to solve a given task, it is important to provide adaptive scaffolding that guides the student towards a solution. We introduce a scaffolding framework based on pop quizzes presented as multi-choice programming tasks. To automatically generate these pop quizzes, we propose a novel algorithm, PQUIZSYN. More formally, given a reference task with a solution code and the student's current attempt, PQUIZSYN synthesizes new tasks for pop quizzes with the following features: (a) Adaptive (i.e., individualized to the student's current attempt), (b) Comprehensible (i.e., easy to comprehend and solve), and (c) Concealing (i.e., do not reveal the solution code). Our algorithm synthesizes these tasks using techniques based on symbolic reasoning and graphbased code representations. We show that our algorithm can generate hundreds of pop quizzes for different student attempts on reference tasks from Hour of Code: Maze Challenge [11] and Karel [9]. We assess the quality of these pop quizzes through expert ratings using an evaluation rubric. Further, we have built an online platform for practicing block-based programming tasks empowered via pop quiz based feedback, and report results from an initial user study.

Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models

“…P b along the following binary attributes: (i) P f is syntactically correct and is obtained by making a small number of edits to fix P b ; (ii) X is complete, i.e., contains adequate information about all errors and required fixes; (iii) X is correct, i.e., the provided information correctly explains errors and required fixes; (iv) X is comprehensible, i.e., easy to understand, presented in a readable format, and doesn't contain redundant information. These attributes are inspired by evaluation rubrics used in literature [6,[25][26][27]. In our evaluation, feedback quality is evaluated via ratings by experts along these four attributes.…”