Transfer-Learning Methods in Programming Course Outcome Prediction

Abstract

The computing education research literature contains a wide variety of methods that can be used to identify students who are either at risk of failing their studies or who could benefit from additional challenges. Many of these are based on machine-learning models that learn to make predictions based on previously observed data. However, in educational contexts, differences between courses set huge challenges for the generalizability of these methods. For example, traditional machine-learning methods assume identical distribution in all data—in our terms, traditional machine-learning methods assume that all teaching contexts are alike. In practice, data collected from different courses can be very different as a variety of factors may change, including grading, materials, teaching approach, and the students. Transfer-learning methodologies have been created to address this challenge. They relax the strict assumption of identical distribution for training and test data. Some similarity between the contexts is still needed for efficient learning. In this work, we review the concept of transfer learning especially for the purpose of predicting the outcome of an introductory programming course and contrast the results with those from traditional machine-learning methods. The methods are evaluated using data collected in situ from two separate introductory programming courses. We empirically show that transfer-learning methods are able to improve the predictions, especially in cases with limited amount of training data, for example, when making early predictions for a new context. The difference in predictive power is, however, rather subtle, and traditional machine-learning models can be sufficiently accurate assuming the contexts are closely related and the features describing the student activity are carefully chosen to be insensitive to the fine differences.

Introductory programming course: review and future implications

“…McCall & Kölling, 2019; Ullah et al, 2019;Ureel & Wallace, 2019;Albluwi, 2018;Bhatia, Kohli & Singh, 2018;Lagus et al, 2018;Lin et al, 2018;Turner, Pérez-Quiñones & Edwards, 2018;Wainer & Xavier, 2018;Castro-Wunsch, Ahadi & Petersen, 2017;Ahadi, Hellas & Lister, 2017;Echeverría et al, 2017;Hsiao, Huang & Murphy, 2017;Malliarakis, Satratzemi & Xinogalos, 2016;Scott et al, 2015;Xinogalos, 2015;Allinjawi, Al-Nuaim & Krause, 2014;Omer, Farooq & Abid, 2020 Greater or equal to 4 and less than 5 18Azcona, Hsiao & Smeaton, 2018;Esteero et al, 2018;Gomes & Correia, 2018;King, 2018;Koong et al, 2018;Premchaiswadi, Porouhan & Premchaiswadi, 2018;Dorodchi, Dehbozorgi & Frevert, 2017;España-Boquera et al, 2017;Estey, Keuning & Coady, 2017;Fu et al, 2017;Kumar, 2017;Seanosky et al, 2017;Seeling & Eickholt, 2017;Gomes, Correia & Abreu, 2016;Seeling, 2016;Simkins & Decker, 2016;Ahadi et al, 2015;Ashenafi, Riccardi & Ronchetti, 2015;Carter, Hundhausen & Adesope, 2015;Hilton & Rague, 2015;Janke, Br...…”

“…Moreover, the effect of changing the sequence of teaching different programing concepts was investigated (Janke, Brune & Wagner, 2015). In addition, a transfer learning approach was evaluated for examining the improvements in prediction accuracies (Lagus et al, 2018). This approach was proposed to manage the heterogeneity of data by Choice of concepts to solve programming problem (recursion or iteration).…”

“…McCall & Kölling, 2019;Azcona, Hsiao & Smeaton, 2018;Pereira et al, 2020;Chaweewan et al, 2018;Omer, Farooq & Abid, 2020;Ninrutsirikun et al, 2020;Premchaiswadi, Porouhan & Premchaiswadi, 2018;Turner, Pérez-Quiñones & Edwards, 2018;Ahadi, Hellas & Lister, 2017;Ahmad et al, 2017;al-Rifaie, Yee-King & d'Inverno, 2017;Carter, Hundhausen & Adesope, 2017;Castro-Wunsch, Ahadi & Petersen, 2017;Chung & Hsiao, 2020;Funabiki, Ishihara & Kao, 2016;Watson, Li & Godwin, 2014;Ahadi et al, 2015;Ashenafi, Riccardi & Ronchetti, 2015;Carter, Hundhausen & Adesope, 2015 Approach examination Liao et al, 2019;King, 2018;Lagus et al, 2018;Lin et al, 2018;Santana, Figueredo & Bittencourt, 2018;Effenberger, Pelánek & Čechák, 2020;Echeverría et al, 2017;España-Boquera et al, 2017;Iqbal Malik & Coldwell-Neilson, 2017;Seeling & Eickholt, 2017;Malliarakis, Satratzemi & Xinogalos, 2016;Seeling, 2016;Hilton & Rague, 2015;Janke, Brune & Wagner, 2015;Rubio et al, 2014;Rosiene & Rosiene, 2015;Scott et al, 2015;Wood & Keen, 2015;…”

Pairing-Based Approach to Support Understanding of Object-Oriented Concepts and Programming

“…Besides, Marin et al [8] propose the use of gamification in programming courses. While Lagus et al [9] suggest transfer-learning methods to predict programming course outcomes. Bosnić et al [10] studied the impact of the distributed software development course among software engineering fresh graduates.…”

Predictive Models for Imbalanced Data: A School Dropout Perspective

“…Among EDM's applications, prediction of school performance and dropout has been gaining prominence since it detects a possible dropout or failure in academic activity [3][4][5][6][7]. So, it is possible to intervene and avoid low performance, or even the student evasion.…”