Targeting At-risk Students Using Engagement and Effort Predictors in an Introductory Computer Programming Course

Abstract

Abstract. This paper presents a new approach to automatically detecting lower-performing or "at-risk" students on computer programming modules in an undergraduate University degree course. Using historical data from previous student cohorts we built a predictive model using logged interactions between students and online resources, as well as students' progress in programming laboratory work. Predictions were calculated each week during a 12-week semester. Course lecturers received student lists ranked by their probability of failing the next computerbased laboratory exam. At-risk students were targeted and offered assistance during laboratory sessions by the lecturer and laboratory tutors. When we group students into two cohorts depending on whether they failed or passed their first laboratory exam, the average margin of improvement on the second laboratory exam between the higher and lowerperforming students was four times higher when our predictions were run and subsequent laboratory support targeted at these students, compared to students from the year our model was trained on.

user2code2vec

“…This allowed researchers and Faculty to gather a fine-grained digital footprint of students learning programming at our University [5]. Recently, research in Learning Analytics has focused on Predictive Modelling and identifying those students having difficulties with course material, also in programming courses [9], and offering remediation, personalized feedback and interventions to students using Machine Learning techniques [6,8]. Prior work has reported that customized notifications sent to students regarding their performance and offering resources such as further learning material, code solutions from peers in their class and university support services helped students to increase their differential performance and engagement on these programming courses [7].…”

Personalizing Computer Science Education by Leveraging Multimodal Learning Analytics

“…However, after identifying those "at-risk" students, we should intervene and help those students. Targeting these weak students during laboratory sessions can aid some students [2], but lecturers usually do not have the time or resources to support many students in large classes or to spend as much time identifying what the student knows and does not know. Automatic interventions for programming classes are having great success in other institutions and environments and we are eager to develop our own strategies using our platforms and resources.…”

“…Laboratory sessions and computer-based examinations are carried out during the teaching period. Our previous study [2] was done on Computer Programming I, CS1, that introduces firstyear students to computer programming and the fundamentals of computational problem solving during their first semester. We work with the following courses that are taught in the second semester (Fall): , that was taught to second-year EC students for the first time on the first semester.…”

“…To clarify, classifiers from week 1 to 6 were trained to predict the mid-semester's laboratory exam's outcome (pass or fail for each student) and from 7 to 12 the end-of-semester's laboratory exam's outcome. The Empirical Error Minimization (EMR) approach has been employed to determine the learning algorithm with the fewest empirical errors from a bag of classifiers C [11,2]. On these models, instead of taking the learning algorithm with the lowest empirical risk or the highest metric (namely accuracy or F1-score), we looked at these metrics per class (pass or fail).…”

“…PredictCS is a Predictive Analytics platform for Computer Science courses that notifies students based on their performance using past student data and recommends most suitable resources for students to consult [1]. The first implementation of this framework was on an introductory computer programming course in 2017 [2]. This implementation trained a model of student performance using one year of groundtruth with data features including engagement and programming effort.…”