Using Data Mining for the Early Identification of Struggling Learners in Physician Assistant Education

Abstract

Purpose Despite the importance of early intervention and remediation, the relatively short duration of physician assistant education programs necessitates the importance of early identification of at-risk learners. This study sought to ascertain whether machine learning was more effective than logistic regression in predicting remediation status among students, using the limited set of data available before or immediately following the first semester of study as predictor variables and academic remediation as an outcome variable. Methods The analysis included one institution and student data from 177 graduates between 2017 and 2019. We employed one data mining model, random forest trees, and compared it to a traditional predictive analysis method, logistic regression. Due to the small sample size, we employed leave-one-out cross-validation and bootstrap aggregation. Results Data provided evidence that the random forest algorithm correctly identified individuals who would later experience academic intervention with a 63.3% positive predictive value, whereas logistic regression exhibited a positive predictive value of 16.6%. Conclusions This single-institution study indicates that predictive modeling, employing machine learning, may be a more effective means than traditional statistical methods of identifying and providing assistance to learners who may experience academic challenges.

Optimization of Ideological and Political Education Management Strategies under k-Means Algorithm in Big Data Environment

“…e preference model was constructed by using the ontology, the semantic relationship among knowledge was better understood, and the interest of students in learning was discovered. Black et al (2021) [12] analyzed the academic performance and behaviour of some engineering students and collected data from score tables and other related factors. e final model for two datasets was constructed under decision trees and naive Bayes algorithms, and the model could be used to predict the performance of students accurately.…”

The Application of Adaptive Minimum Match k-Nearest Neighbors to Identify At-Risk Students in Health Professions Education

“…11 Within health professions specifically, commentaries on machine learning abound, [12][13][14] whereas empirical studies that apply machine learning are less common. Black et al 15 take a similar approach to ours within the PA education context (which we built on by using additional predictive models, validating our results on an entire new cohort, and addressing practical applications of analytics results). Predictive analytic methods have been used on data from students learning oral pathology, 16 blended medicine, 17 and psychomotor skills.…”

“…We evaluated the results of all predictive models using leave-one-out cross-validation (LOOCV), which has been used or recommended previously for similar situations with small sample sizes. 15,30 We adhered as closely as possible to recommendations from Rao et al 31 For each predictive model, to execute LOOCV in our sample of 181 students, we trained the model on 180 students and tested it on the remaining one student. We repeated this process 181 times such that each student was the testing student one time.…”

The application of adaptive minimum match k-nearest neighbors to identify at-risk students in health professions education

“…Within health professions education specifically, Black et al (2021) have taken a similar approach within the physician assistant education context. Predictive analytic methods have been used on data from students learning oral pathology (Walkowski et al 2015), blended medicine (Saqr et al 2017), and psychomotor skills (Chan et al 2020).…”

“…We evaluated the results of all predictive models using leave-one-out cross validation (LOOCV), which has been used or recommended before in similar situations with small sample sizes (Zohair 2019; Black et al 2021). We adhered as closely as possible to recommendations from Rao et al (2008).…”