Student Engagement Predictions in an e-Learning System and Their Impact on Student Course Assessment Scores

Abstract

Several challenges are associated with e-learning systems, the most significant of which is the lack of student motivation in various course activities and for various course materials. In this study, we used machine learning (ML) algorithms to identify low-engagement students in a social science course at the Open University (OU) to assess the effect of engagement on student performance. The input variables of the study included highest education level, final results, score on the assessment, and the number of clicks on virtual learning environment (VLE) activities, which included dataplus, forumng, glossary, oucollaborate, oucontent, resources, subpages, homepage, and URL during the first course assessment. The output variable was the student level of engagement in the various activities. To predict low-engagement students, we applied several ML algorithms to the dataset. Using these algorithms, trained models were first obtained; then, the accuracy and kappa values of the models were compared. The results demonstrated that the J48, decision tree, JRIP, and gradient-boosted classifiers exhibited better performance in terms of the accuracy, kappa value, and recall compared to the other tested models. Based on these findings, we developed a dashboard to facilitate instructor at the OU. These models can easily be incorporated into VLE systems to help instructors evaluate student engagement during VLE courses with regard to different activities and materials and to provide additional interventions for students in advance of their final exam. Furthermore, this study examined the relationship between student engagement and the course assessment score.

Systematic review of research on artificial intelligence applications in higher education – where are the educators?

“…Student models and academic achievement Many more studies are concerned with profiling students and modelling learning behaviour to predict their academic achievements at the course level. Hussain et al (2018) applied several machine learning algorithms to analyse student behavioural data from the virtual learning environment at the Open University UK, in order to predict student engagement, which is of particular importance at a large scale distance teaching university, where it is not possible to engage the majority of students in face-to-face sessions. The authors aim to develop an intelligent predictive system that enables instructors to automatically identify low-engaged students and then to make an intervention.…”

“…Evaluation of student understanding, engagement and academic integrity Three articles reported on student-facing tools that evaluate student understanding of concepts (Jain, Gurupur, Schroeder, & Faulkenberry, 2014;Zhu, Marquez, & Yoo, 2015) and provide personalised assistance (Samarakou, Fylladitakis, Früh, Hatziapostolou, & Gelegenis, 2015). Hussain et al (2018) used machine learning algorithms to evaluate student engagement in a social science course at the Open University, including final results, assessment scores and the number of clicks that students make in the VLE, which can alert instructors to the need for intervention, and Amigud, Arnedo-Moreno, Daradoumis, and Guerrero-Roldan (2017) used machine learning algorithms to check academic integrity, by assessing the likelihood of student work being similar to their other work. With a mean accuracy of 93%, this opens up possibilities of reducing the need for invigilators or to access student accounts, thereby reducing concerns surrounding privacy.…”

Data Mining for the e-Learning Risk Management

“…The different types of methodologies for analysis of the relations are used: search for strong rules, analysis of temporary patterns, correlation data mining, analysis of causes (Apurva 2017) Data processing for assessing Authentication and evaluation of data and judgment. Compilation, visualization and interactive interface to support decision making are used (Rawat 2019;Hussain 2018) Outlier detection Data discovery which deviate greatly from remaining values (Panyajamorn 2018) Text mining…”

Online module login data as a proxy measure of student engagement: the case of myUnisa, MoyaMA, Flipgrid, and Gephi at an ODeL institution in South Africa

“…ENG2601 students' myUnisa module-level login file access metrics as visualised by Gephi represent their engagement -especially behavioural engagement as a component of a multi-dimensional view of student engagement -with their module, Applied English Language Studies: Further Explorations (ENG2601). That is, these metrics serve as a proxy measure of these students' engagement with this module (Badge et al, 2012;Beer et al, 2010;Boulton et al, 2018;Hernández-García et al, 2016;Hussain et al, 2018). In this regard, Badge et al (2012) point out that in their study that used Friendfeed, network data extracted and visually represented using Gephi offered an uncomplicated but effective proxy for student engagement.…”

“…With the three highest file access frequency counts shared by three participants in the HAU sub-cohort, these three participants also shared almost half the student engagement metrics when compared to the other participants in this sub-cohort. In relation to using access to LMS activities, which in this case are module-level file access frequencies, as a measurement of student engagement, Hussain et al (2018) contend that in online learning environments (OLEs), it is not easy to capture a student's engagement through conventional methodologies like class attendance as such methodologies are not directly available on LMSs. In a different but related context, Beer et al (2010) and Holmes (2018) point out that monitoring and capturing student interaction with learning materials on LMSs is one mode of tracing student engagement.…”