Review of Research on Student-Facing Learning Analytics Dashboards and Educational Recommender Systems

Abstract

This article is a comprehensive literature review of student-facing learning analytics reporting systems that track learning analytics data and report it directly to students. This literature review builds on four previously conducted literature reviews in similar domains. Out of the 945 articles retrieved from databases and journals, 93 articles were included in the analysis. Articles were coded based on the following five categories: functionality, data sources, design analysis, student perceptions, and measured effects. Based on this review, we need research on learning analytics reporting systems that targets the design and development process of reporting systems, not only the final products. This design and development process includes needs analyses, visual design analyses, information selection justifications, and student perception surveys. In addition, experiments to determine the effect of these systems on student behavior, achievement, and skills are needed to add to the small existing body of evidence. Furthermore, experimental studies should include usability tests and methodologies to examine student use of these systems, as these factors may affect experimental findings. Finally, observational study methods, such as propensity score matching, should be used to increase student access to these systems but still rigorously measure experimental effects.

SRLx: A Personalized Learner Interface for MOOCs

“…Providing learners with visualizations of their progress enables them to reflect upon their learning, and an emerging body of research has begun to empirically evaluate the effectiveness of such feedback [1,2,6,10]. Over time, this reflection should improve learners' use of SRL strategies [3,8].…”

Curricular Concept Maps as Structured Learning Diaries: Collecting Data on Self-Regulated Learning and Conceptual Thinking for Learning Analytics Applications

“…Meta-analytical evidence suggests that properly designed SRL interventions can improve learning (Dignath & Büttner, 2008;Panadero, 2017;Sitzmann & Ely, 2011). Interventions to enhance students' SRL include, for example, self-assessment (Panadero & Alonso-Tapia, 2013;Panadero, Jonsson, & Botella, 2017), concept mapping (Gurlitt & Renkl, 2010;Nesbit & Adesope, 2006;Novak & Musonda, 1991), learning diaries (Klug, Ogrin, Keller, Ihringer, & Schmitz, 2011;Schmitz & Wiese, 2006), and learning analytics dashboards (Bodily & Verbert, 2017;Jivet et al, 2017;Sedrakyan et al, 2018). In this study, we are at the intersection of this knowledge base.…”

“…Research literature related to learning analytics applications reveals that surprisingly little consideration has been given to which data is meaningful to collect; this is underlined by Winne's (2017) notion that out of several widely cited descriptions of learning analytics, none answer the question of which data should be gathered for input to methods that generate learning analytics. In a review of student-facing dashboards, Bodily & Verbert (2017) found that the most often used types of data in dashboards were (1) the number of times a resource was accessed (in 75% of articles), (2) data on student mastery as measured by assessment (in 37% of articles), and (3) data on time spent accessing resources (in 30% of articles). It is clear that the availability of data may affect which data is used in learning analytics.…”

Ethical Learning Analytics: “Do No Harm” Versus “Do Nothing”

“…First, as we have discussed in the IU and UMBC examples, we support student facing analytics or dashboards (Bodily & Verbert, 2017) that provide students with a perspective on where they currently stand, taking the approach of providing students with tools to orient them vs. a projected trajectory of predicted outcomes not yet realized. In fact, we like to say that predictive analytics should be used to identify at‐risk students, but never as a means to motivate them—by sharing the prediction as a fait accompli .…”