Using Educational Analytics to Improve Test Performance

Abstract

Abstract. Learning analytics are being used in many educational applications in order to help students and Faculty. In our work we use predictive analytics, using student behaviour to predict the likely performance of end of semester final grades with a system we call PredictED. The main contribution of our approach is that our intervention automatically emailed students on a regular basis, with our prediction for the outcome of their exam performance. We targeted first year, first semester University students who often struggle with making the transition into University life where they are given much more responsibility for things like attending class, completing assignments, etc. The form of student behaviour that we used is students' levels and types of engagement with the University's Virtual Learning Environment (VLE), Moodle. We mined the Moodle access log files for a range of parameters based on temporal as well as content access, and use machine learning techniques to predict likely pass/fail, on a weekly basis throughout the semester using logs and outcomes from previous years as training material. We chose ten first-year modules with reasonably high failure rates, large enrolments and stability of module content across the years to implement an early warning system on. From these modules 1,558 students were registered for one of these modules. They were offered the chance to opt into receiving weekly email alerts warning them about their likely outcome. Of these 75% or 1,181 students opted into this service. Pre-intervention there were no differences between participants and non-participants on a number of measures related to previous academic record. However, postintervention the first-attempt final grade performance yielded nearly 3% improvement (58.4% to 61.2%) on average for those who opted in. This tells us that providing weekly guidance and personalised feedback to vulnerable first year students, automatically generated from monitoring of their online behaviour, has a significant positive effect on their exam performance.

Student Engagement Predictions in an e-Learning System and Their Impact on Student Course Assessment Scores

“…The AUC ranges from a minimum of 0 to a maximum of 1. When the AUC of a model is greater than 0.5, it is considered a good model [ 22 ].…”

“…A predictive system enables an instructor to automatically identify low-engagement students during a course based on activities from that online course. Given such detection, the instructor can then motivate (e.g., send an e-mail reminder) or identify difficulties during the course [ 22 ]. When a student receives an advisory e-mail from an instructor (i.e., an e-mail asking about any difficulty), on a weekly basis, the student is more likely to work hard and increase their engagement.…”

Personalizing Computer Science Education by Leveraging Multimodal Learning Analytics

“…In addition, Crowd::Debug [7] was presented as a similar solution for Ruby, a test-driven development language. In terms of notifying students how they are progressing throughout a semester, Purdue University's Course Signals [8] sends a personalised mail and posts a traffic signal as an indicator of their performance while Dublin City University's PredictED [9] project notifies students how they are doing and where they are within their own class. Both systems yielded impressive improvement in first-year retention rates.…”

Innovative Learning Analytics Research at a data-driven HEI

“…We also observed a 3.5% improvement in actual exam mark between those who opted into the study and those who chose not to. (Corrigan (2015)). Once it is demonstrated that it is possible to predict students' examination grades with a reasonable level of accuracy, it is important to use this to help students in some way.…”