Predicting new student performances and identifying important attributes of admission data using machine learning techniques with hyperparameter tuning

Abstract

Recently, many global universities have faced high student failure and early dropout rates reflecting on the quality of education. To tackle this problem, forecasting student success as early as possible with machine learning is one of the most important approaches used in modern universities. Thus, this study aims to analyze and compare models for the early prediction of student performance with six machine learning based on Thailand’s education curriculum. A large dataset was collected from the admission scores of 5,919 students during 2011-2021 of 10 programs in the Faculty of Science at Ubon Ratchathani University. The methodology was carried out using Jupyter Notebook, Python 3, and Scikit-Learn to build the models for prediction. To obtain a higher result, we needed not only to find high-performance prediction models, but also to tune hyperparameter configurations consisting of 138 possible different patterns to identify the best-tuned model for each classifier. Furthermore, we investigated significantly important predictors affecting student success for 10 programs in our faculty. In the experiments, the process was divided into two parts: First, we evaluated effective models using a confusion matrix with 10-fold cross-validation. The results showed that random forest (RF) had the highest F1-measure of 86.87%. While predictive models using fine-tuned RF of 10 programs claimed accuracy of about 72% to 93%. Second, we computed the importance of each feature with fine-tuned RF classifiers. The result showed that national test scores (e.g., ONET-English, ONET-Math, ONET-Science, ONET-Social studies, ONET-Thai, and PAT2), entry type, and school grade (e.g., art, English, GPA, health, math, science, and technology) are highly influential features for predicting student success. In summary, these results yield many benefits for other relevant educational institutions to enhance student performance, plan class strategies and undertake decision-making processes.