Multimodal learning analytics—In‐between student privacy and encroachment: A systematic review

Abstract

Since the emergence of learning analytics (LA) in 2011 as a distinct field of research and practice, multimodal learning analytics (MMLA), shares an interdisciplinary approach to research and practice with LA in its use of technology (eg, low cost sensors, wearable technologies), the use of artificial intelligence (AI) and machine learning (ML), and the provision of automated, mostly real‐time feedback to students and instructors. Much of MMLA takes place in experimental and laboratory settings, researching students' learning in in‐between spaces—between research and classroom application, in‐between students' learning in private and public spaces as researchers track students' learning both in their use of social media and connected devices, and through the use of context‐aware and adaptive devices; and lastly, in‐between respecting students' privacy while increasingly using intrusive technologies. This study seeks to establish what is known about MMLA in terms of rationale for applications, the nature and scope of data collected, the study contexts, evidence of commercial interests and/or downstream uses of students' data, and consideration of ethics, privacy, and the protection of student data. This systematic review analysed 192 articles using a search string consisting of various combinations of multimodal (data) and learning analytics. The main findings include, inter alia, that though MMLA provide insights into learning and teaching, there is little evidence of MMLA findings successfully being applied to classroom settings, at scale. Given that the nature of MMLA research often necessitates the use of a range of (intrusive) sensors and recording technologies and can include children in its samples; the encroachment of students' right to privacy is a huge concern that is not addressed. There is also a need to reconsider the rationale for collecting multimodal data, the conditions under which such data collection will be ethical and in service of students' wellness, and the boundaries that should protect their (multimodal) data. Practitioner notesWhat is already known about this topic Experimental educational research predates both multimodal learning analytics (MMLA) and educational data mining (EDM). MMLA has been an integral part of learning analytics as research focus and practice since its inception. The increased digitisation and datafication, advances in technology (eg, sensor technology, geo‐tracking, etc.) and a growing normalisation of wearable technologies provides greater scope for collecting multimodal data. There is a vast body of published MMLA research providing a range of insight into students' and educators' behaviours aiming to increase the effectiveness of teaching. What this paper adds While there are several studies providing insights into the state of MMLA research, this study provides insight to a selected range of factors in MMLA research such as the type of research (empirical/conceptual); the nature and scope of data collected; the sample populations (pre‐higher education, higher education, etc.); evidence of commercial interests or consideration of downstream uses; issues pertaining to consent, privacy, data protection and ethics; and evidence of how findings were used to improve teaching and learning. The vast majority of MMLA research targets higher education, is empirical in nature and is based on relatively small samples of participation in experimental settings. Confirms previous research that found the predominance of small samples, and a lack of replicability and, as a result, lacking scalability. That there is very little explicit discussion of the ethical and privacy implications and data protection, either at design stage or for future implementation. Similarly, that there is little consideration of potential commercial interests or downstream uses of data. Implications for practice and/or policy MMLA in its essence requires interdisciplinary approaches and teams. For MMLA to move beyond small‐scale, experimental settings to application in real (classroom) settings, larger, replicable studies should be conducted together with ways to make study findings actionable for teachers and students. Ethical issues, commercial interests and downstream uses of collected data must be considered within the design and approval of MMLA research.