On predicting academic performance with process mining in learning analytics

Abstract

Purpose The purpose of this paper is to propose a process mining approach to help in making early predictions to improve students’ learning experience in massive open online courses (MOOCs). It investigates the impact of various machine learning techniques in combination with process mining features to measure effectiveness of these techniques. Design/methodology/approach Student’s data (e.g. assessment grades, demographic information) and weekly interaction data based on event logs (e.g. video lecture interaction, solution submission time, time spent weekly) have guided this design. This study evaluates four machine learning classification techniques used in the literature (logistic regression (LR), Naïve Bayes (NB), random forest (RF) and K-nearest neighbor) to monitor weekly progression of students’ performance and to predict their overall performance outcome. Two data sets – one, with traditional features and second, with features obtained from process conformance testing – have been used. Findings The results show that techniques used in the study are able to make predictions on the performance of students. Overall accuracy (F1-score, area under curve) of machine learning techniques can be improved by integrating process mining features with standard features. Specifically, the use of LR and NB classifiers outperforms other techniques in a statistical significant way. Practical implications Although MOOCs provide a platform for learning in highly scalable and flexible manner, they are prone to early dropout and low completion rate. This study outlines a data-driven approach to improve students’ learning experience and decrease the dropout rate. Social implications Early predictions based on individual’s participation can help educators provide support to students who are struggling in the course. Originality/value This study outlines the innovative use of process mining techniques in education data mining to help educators gather data-driven insight on student performances in the enrolled courses.

Systematic review of research on artificial intelligence applications in higher education – where are the educators?

“…Table 7 shows that machine learning methods outperformed logistic regression in all studies in terms of their classification accuracy in percent. To evaluate the performance of classifiers, the F1-score can also be used, which takes into account the number of positive instances correctly classified as positive, the number of negative instances incorrectly classified as positive, and the number of positive instances incorrectly classified as negative (Umer et al, 2017; for a brief overview of measures of diagnostic accuracy, see Šimundić, 2009). The F1-score ranges between 0 and 1 with its best value at 1 (perfect precision and recall).…”

Modelling a Machine Learning Classifier for Predicting Student’s Entrepreneurial Intentions

“…Aluko, Daniel, Shamsideen Oshodi, Aigbavboa, & Abisuga, [3] compared the prediction accuracy of Support vector machine and logistic regression using a sample size of 102 architecture students for predicting the performance of students in academics and found that Support vector machine classifier is better than logistic regression in predicting students' academic performance. Umer, Susnjak, Mathrani, & Suriadi, [14] have used three Machine learning classification algorithms to predict the performance of students by recording their weekly performance in MOOCS environment. The Study revealed that combining process mining with the standard machine learning techniques improved the accuracy of the models.…”

Process Mining Contributions to Discrete-event Simulation Modelling

“…On a similar note, Mukala et al (2015) use process mining techniques to monitor and analyse student learning habits based on data collected from Massive Open Online Courses (MOOCs) establishing that successful students always watch videos in the recommended sequence, while the opposite is true for unsuccessful students. Related to MOOCs, Umer et al (2017) propose a process mining approach to enhance the student learning experience by combining different machine learning techniques with process mining features to measure the effectiveness of different techniques. The ultimate aim of that particular data-based approach is to help improve the student learning experience and reduce dropout rates in MOOCs.…”