The efficacy of learning analytics interventions in higher education: A systematic review

Abstract

Educational institutions are increasingly turning to learning analytics to identify and intervene with students at risk of underperformance or discontinuation. However, the extent to which the current evidence base supports this investment is currently unclear, and particularly so in relation to the effectiveness of interventions based on predictive models. The aim of the present paper was to conduct a systematic review and quality assessment of studies on the use of learning analytics in higher education, focusing specifically on intervention studies. Search terms identified 689 articles, but only 11 studies evaluated the effectiveness of interventions based on learning analytics. These studies highlighted the potential of such interventions, but the general quality of the research was moderate, and left several important questions unanswered. The key recommendation based on this review is that more research into the implementation and evaluation of scientifically driven learning analytics is needed to build a solid evidence base for the feasibility, effectiveness, and generalizability of such interventions. This is particularly relevant when considering the increasing tendency of educational institutions around the world to implement learning analytics interventions with only little evidence of their effectiveness.

Utilising learning analytics to support study success in higher education: a systematic review

“…A recent systematic review of 11 publications on the efficacy of learning analytics interventions in higher education documents visual signals and other dashboard features as dominant elements (Larrabee Sønderlund et al 2018). Beyond these findings, our systematic review adds further insight into interventions for supporting study success (see Table 1).…”

“…From an instructional design perspective, visualisations, features, and interventions currently being implemented in learning analytics dashboards are of a simplistic nature, i.e., they provide statistics which are less informative for supporting learning processes (He et al 2015). As documented by Larrabee Sønderlund et al (2018) and in this systematic review, learning analytics interventions need to enable active learning, such as through adaptive scaffolds or by helping teachers to curate and act on data about their students (Arthars et al 2019;Darlington 2017;Dawson et al 2017). This includes the provision of a better understanding of students' expectations of learning analytics features to support learning processes and warrant study success (Schumacher and Ifenthaler 2018).…”

“…Thus, the increased importance of data in education has led to an upsurge in primary research publications in learning analytics (Prieto et al 2019), which indicate that the analysis of digital traces of learning and teaching may reveal benefits for learners, teachers, learning environments, or the organisation (Gašević et al 2015). Although there are initial systematic reviews on learning analytics, such as on policy recommendations for learning analytics (Ferguson et al 2016), identification of learning analytics research objectives and challenges (Papamitsiou and Economides 2014), learning analytics in the context of distance education (Kilis and Gulbahar 2016), and more recently on the efficiency of learning analytics interventions (Larrabee Sønderlund et al 2018), there exists no current and comprehensive systematic review focusing on learning analytics for supporting study success.…”

Challenges and Future Directions of Big Data and Artificial Intelligence in Education

“…Only a small number of studies have focused on the effectiveness of learning analytics programs and AI applications. However, recent findings have revealed encouraging results in terms of improving students' academic performance and retention, as well as supporting teachers in learning design and teaching strategy refinement (Viberg et al, 2018;Li et al, 2019;Sonderlund et al, 2019;Mislevy et al, 2020).…”

“I’m Not a Number, I’m Someone to Them”: Supporting Commencing University Students’ Through Technology-Mediated Personalised Communication

“…The field of LA has emerged as a way to capitalise on the availability of large datasets, to bring a richer understanding of student learning and student engagement (Clow, 2013). A nascent development in education, reviews have already featured the promises of LA interventions for improving student learning (e.g., Sclater et al, 2016;Sønderlund et al, 2019).…”