Optimized Screening for At-Risk Students in Mathematics: A Machine Learning Approach

Abstract

Traditional screening approaches identify students who might be at risk for academic problems based on how they perform on a single screening measure. However, using multiple screening measures may improve accuracy when identifying at-risk students. The advent of machine learning algorithms has allowed researchers to consider using advanced predictive models to identify at-risk students. The purpose of this study is to investigate if machine learning algorithms can strengthen the accuracy of predictions made from progress monitoring data to classify students as at risk for low mathematics performance. This study used a sample of first-grade students who completed a series of computerized formative assessments (Star Math, Star Reading, and Star Early Literacy) during the 2016–2017 (n = 45,478) and 2017–2018 (n = 45,501) school years. Predictive models using two machine learning algorithms (i.e., Random Forest and LogitBoost) were constructed to identify students at risk for low mathematics performance. The classification results were evaluated using evaluation metrics of accuracy, sensitivity, specificity, F1, and Matthews correlation coefficient. Across the five metrics, a multi-measure screening procedure involving mathematics, reading, and early literacy scores generally outperformed single-measure approaches relying solely on mathematics scores. These findings suggest that educators may be able to use a cluster of measures administered once at the beginning of the school year to screen their first grade for at-risk math performance.

Data mining approach for prediction of academic success in open and distance education

“…Researchers have conducted numerous studies to achieve a more comprehensive understanding of student success and academic performance. These studies have focused on factors influencing student achievement at the university level (Taşdemir, 2012), comparing academic success in open education (Tosun, 2016), predicting students at risk within higher education (Apaydın et al, 2020), and identifying the key factors impacting academic performance alongside demographic variables, including the identification of the most influential variables for predicting mathematics performance (Bulut et al, 2022).…”

“…in data mining analyses were found to be effective in students' academic performance. Bulut et al (2022) built prediction models using Random Forest and LogitBoost algorithms to identify at-risk students for low mathematics performance. Nahar et al (2021) compared six classification algorithms (random forest, DT(J48), Naive Bayes, PART bagging, boosting) in their study on improving academic achievement and created two final models based on decision tree and Naive Bayes algorithms for two of the data sets.…”

“…Within the scope of this study, it is aimed to discover the factors affecting the academic success of students taking compulsory courses between 2011-2017 by using data mining methods. Based on the finding that academic measurements will be an influential variable in producing the most helpful information for the prediction of academic success (Bulut et al, 2022), the academic success status of students in different programs was tried to be predicted with the course success grades of their compulsory courses, the type of program they enrolled in and their demographic features. This approach is considered a valuable method for developing policies to support students with low academic performance or to encourage advanced students, especially from the first semester.…”

“…The negative predictive value is the ratio of actual negative cases to the cases the model classifies as negative. Another criterion, MCC, which is less common in the literature than other criteria, is more successful, especially in comparing algorithms in unbalanced classes (Bulut et al, 2022). As values other than F1 near 1, it indicates perfect classification.…”