Prutor: A System for Tutoring CS1 and Collecting Student Programs for Analysis

Abstract

An introductory programming course (CS1) is an integral part of any undergraduate curriculum. Due to large number and diverse programming background of students, providing timely and personalised feedback to individual students is a challenging task for any CS1 instructor. The help provided by teaching assistants (typically senior students) is not sufficient as it suffers from unintentional bias and, most of the time, not quick enough.In this paper, we present Prutor, a tutoring system platform to conduct introductory programming courses. Prutor is a cloud-based web application that provides instant and useful feedback to students while solving programming problems. Prutor stores, at regular intervals, the snapshots of students' attempts to solve programming problems. These intermediate versions of the student programs provide the instructors (and data analysts) a view of the students' approach to solving programming problems. Since Prutor is accessible through any standard web browser, students do not need to worry about dependencies external to the programming course, viz. Operating Systems, Editors, Compilers, Compiler Options, etc.. This enables the students to focus on solving only the programming problems. Apart from the code snapshots at regular intervals, Prutor also collects other valuable data such as the time taken by the students to solve the problems, the number of compile and execution events, and the errors made. We have used this data in developing intelligent tools for giving feedback to students, some of which are described briefly in this paper. This system thus serves as a platform for tutoring as well as data collection for researchers.

Automated Assessment in Computer Science Education: A State-of-the-Art Review

“…ProgEdu [50,252] keeps a history of changes to the source code using Git and stores logging data to provide instructors with various statistics charts from which they can comprehend whether submissions are on time and meet requirements, the distribution of failure types, and the quality of students' code. PRogramming tUTOR (PRUTOR) [62] takes code snapshots at regular intervals and provides visualizations of the code progression of students' programs and how the students approach the programming assignments. Edgar [156] provides a dashboard with multiple visualizations on learners' behavior and assignments.…”

“…Tool / Reference Type Language Availability Interoperability URL Web-CAT [75] WP Multi OS LTI + Eclipse + BlueJ https://web-cat.github.io/ PASS [255] WP C/C++, Java A https://www.cs.cityu.edu.hk/~passweb Kattis [77] WP Multi A https://open.kattis.com ECSpooler [13] WS Multi OS https://github.com/collective/ECSpooler STAGE (CodeCover) [172] PG Java, COBOL OS Moodle plugin http://codecover.org JACK [30,136,227] WP Multi A LTI + Eclipse https://jack-demo.s3.uni-due.de URI Online Judge [32] WP Multi A https://www.urionlinejudge.com.br CodeAssessor [241] WP C/C++ VPL [203] PG Multi OS Moodle plugin https://vpl.dis.ulpgc.es/ CloudCoder [108] WP C/C++, Java, Python, Ruby OS https://github.com/daveho/CloudCoder CodeRunner [60] PG Multi OS Moodle plugin https://github.com/trampgeek/moodle-qtype_coderunner JPLAS [83][84][85] WP 1 Java Eclipse LX [61] WS Multi Modiied Moodle Testovid within Protus [242] WP Java OS https://github.com/ivanpribela/testovid TestMyCode [194] WP Multi OS NetBeans https://github.com/testmycode/tmc-server CodeWorkout [74] WP C++, Java, Python, Ruby OS LTI https://github.com/web-cat/code-workout Arena [184] WP Imp/OO Git JavAssess (ASys) [115] LB Java OS http://personales.upv.es/josilga/ASys CodeOcean [226] WP Multi OS LTI https://github.com/openHPI/codeocean Jutge.org [176] WP Multi OS https://github.com/jutge-org GradeIT (PrUTOR) [62,173] CD C A https://prutor.ai neoESPA [148,150] WP C/C++, Java, Python A http://neoespa.pusan.ac.kr Enki (Mooshak) [170] WP Multi OS LTI + Eclipse https://mooshak2.dcc.fc.up.pt ProgEdu [50,252] CD Java OS Git https://github.com/ProgEdu-Organization/ProgEdu Aristotle [2] ...…”

Targeted Example Generation for Compilation Errors

“…The theme of assignments covered during these 7 weeks was input-output, conditionals, iterations, nestediterations, functions, arrays and matrices. All intermediate code attempts were recorded on Prutor [15], a custom IDE developed for teaching CS1. During this 21 hour (3 hours x 7 weeks) period, TEGCER was deployed to provide live feedback to students, in addition to the error messages reported by compiler.…”

“…During this 21 hour (3 hours x 7 weeks) period, TEGCER was deployed to provide live feedback to students, in addition to the error messages reported by compiler. Fig 1 demonstrates the user-interface of TEGCER integrated with the Prutor IDE [15], and Fig 2a provides the backend system overview. Whenever a student in this course offering encountered a compilation error, we first ran our neural network classifier to predict the error-repair class that it belongs to.…”

“…Figure 1 presents an erroneous code attempt by student, on our custom IDE Prutor [15] with TEGCER feedback tool integrated. The students code, written in the top-right editor pane, suffers from compilation errors in 3 separate lines.…”