Editorial: Augmenting Learning Analytics with Multimodal Sensory Data​

Abstract

The goal of Learning Analytics is to understand and improve learning. However, learning does not always occur through or mediated by a technological system that can collect digital traces. To be able to study learning in non-technology centered environments, several signals, such as video and audio, should be captured, processed and analyzed to produce traces of the actions and interactions of the actors of the learning process. The use and integration of the different modalities present in those signals is known as Multimodal Learning Analytics. This editorial presents a brief introduction to this new variation of Learning Analytics and summarizes the four representative articles included in this special issue. The editorial closes with a small discussion about the current opportunities and challenges in multimodal learning analytics.

Multimodal teaching analytics: Automated extraction of orchestration graphs from wearable sensor data

“…This emergent trend to complement easily available digital traces with data captured from the physical world has been labelled multimodal learning analytics (MMLA). Typical examples of MMLA include text-based and graphic-based content, speech, gesture, and electro-dermal activation (Blikstein & Worsley, 2016;Ochoa & Worsley, 2016). For instance, Ochoa et al (2013) used video, audio, and pen stroke information to discriminate between expert and non-expert students solving mathematical problems.…”

A Conversation between Learning Design and Classroom Observations: A Systematic Literature Review

“…Through observations, we can gather data on individual behaviours, interactions, or the educational setting both in physical and digital spaces [8,25] using multiple machine-and human-driven data collection techniques (such as surveys, interviews, activity tracking, teaching and learning content repositories, or classroom and wearable sensors). Indeed, Multimodal Learning Analytics (MMLA) solutions can be seen as "modern" observational approaches suitable for physical and digital spaces [26], to infer climate in the classroom [27], or to observe technology-enhanced learning [28] or to put in evidence the human and machine-generated data for the design of LA systems [29].…”

Applying IDC theory to education in the Alps region: a response to Chan et al.’s contribution

“…For instance, if one can train a machine learning model with a large set of essays classified by teachers as creative or not, then text mining methods can be used to quite accurately classify and provide student feedback (Woods et al 2017). Moreover, new sensors and powerful computational models enable so-called multimodal analytics, i.e., applying data sciences methods to signals that are not captured by traditional keyboard and mouse such as gestures, gaze, body postures, and electro cardiology (Ochoa and Worsley 2016). Multimodal analytics allow for the correlation of such signals to performance, especially for skills that require a more complex measurement scale than binary.…”