Awareness Is Not Enough: Pitfalls of Learning Analytics Dashboards in the Educational Practice

Abstract

Abstract. It has been long argued that learning analytics has the potential to act as a "middle space" between the learning sciences and data analytics, creating technical possibilities for exploring the vast amount of data generated in online learning environments. One common learning analytics intervention is the learning dashboard, a support tool for teachers and learners alike that allows them to gain insight into the learning process. Although several related works have scrutinised the state-of-the-art in the field of learning dashboards, none have addressed the theoretical foundation that should inform the design of such interventions. In this systematic literature review, we analyse the extent to which theories and models from learning sciences have been integrated into the development of learning dashboards aimed at learners. Our critical examination reveals the most common educational concepts and the context in which they have been applied. We find evidence that current designs foster competition between learners rather than knowledge mastery, offering misguided frames of reference for comparison.

Opportunities and Challenges in Using Learning Analytics in Learning Design

“…Intervening during the run-time of a course can become a reality. But as Jivet et al [49] laid out in a recent study, this objective is only seldom reached and, therefore the LA Dashboard applications often fail in supporting the students and teachers in the way that is intended. An instrument that delivers on demand information enabled experiences of adaptations of the learning experiences and thereby an effect of improved personalization.…”

“…and O2.SO3.). Most recently, Jivet et al [49] conducted a study on pitfalls for LA dashboards and showed that most dashboards only consider the reflection process very roughly They conclude that they are not properly designed to meet the needs of their actual users; the teacher and learners.…”

Profiling sympathetic arousal in a physics course: How active are students?

“…However, a recent literature study analysed the extent to which theories and models from learning sciences have been integrated into the development of learning dashboards aimed at learners. The analysis revealed that very few dashboard evaluations take into account the educational concepts which were used as a theoretical foundation for their design (Jivet, Scheffel, Drachsler, & Specht, ; Jivet, Scheffel, Specht, & Drachsler, ). The metrics of in‐class arousal considered in this study could be incorporated into a new kind of “biosensor learning dashboard” for students to be aware of how active they are and stimulate reflection on how they could increase engagement; moreover, this could enable teachers to monitor the number of active students at any time during class and to realize how these activation levels relate to the different tasks they plan for their instruction.…”

Learning analytics made in France: the METAL project

“…Regarding acceptance, the information displayed has to be understandable and understood by the final user, whoever he/she is, and whatever is his/her context, be compliant with privacy issues, etc. (Jivet et al, 2018). In the literature, dashboards can be dedicated to either teachers (Kuzilek, Hlosta, Herrmannova, Zdrahal, & Wolff, 2015) or learners (Davis, S. K. et al, 2018), and in some cases to both of them (Millecamp, M., Gutierrez, F., Charleer, S., Verbert, K., & De Laet, T., 2018).…”