Classroom Social Signal Analysis

Raca, Mirko; Dillenbourg, Pierre

doi:10.18608/jla.2014.13.16

Cited by 11 publications

(11 citation statements)

References 4 publications

(3 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The physiological data sources include eye-tracking, electroencephalography, facial features and arousal data (HR, blood volume pressure (BVP), electrodermal activity (EDA) and skin temperature). Various combinations of such data sources have been used in the past to explain (Raca & Dillenbourg, 2014) and/or predict (Beardsley, Hernández-Leo, & Ramirez-Melendez, 2018) learning behaviors (Furuichi & Worsley, 2018) and/or performance (Junokas, Lindgren, Kang, & Morphew, 2018).…”

Section: Motivation Of the Research And Research Questionmentioning

confidence: 99%

“…Results shown that low arousal was the predominant state, whereas all students were never in high arousal states in the classroom, at the same moment. In the same context, Raca and Dillenbourg (2014) used the synchronization of students' gaze direction and body postures for predicting their self-reported attention. Attention has been found to be a strong construct of engagement (Kinnealey et al, 2012;Mundy, Acra, Marshall, & Fox, 2006).…”

Section: Related Work: Utilizing Multimodal Data To Predict Learning mentioning

confidence: 99%

See 1 more Smart Citation

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Sharma¹,

Papamitsiou²,

Giannakos³

2019

Brit J Educational Tech

View full text Add to dashboard Cite

Students' on‐task engagement during adaptive learning activities has a significant effect on their performance, and at the same time, how these activities influence students' behavior is reflected in their effort exertion. Capturing and explaining effortful (or effortless) behavior and aligning it with learning performance within contemporary adaptive learning environments, holds the promise to timely provide proactive and actionable feedback to students. Using sophisticated machine learning (ML) algorithms and rich learner data, facilitates inference‐making about several behavioral aspects (including effortful behavior) and about predicting learning performance, in any learning context. Researchers have been using ML methods in a “black‐box” approach, ie, as a tool where the input data is the learner data and the output is a given class from the chosen construct. This work proposes a methodological shift from the “black‐box” approach to a “grey‐box” approach that bridges the hypothesis/literature‐driven (feature extraction) “white‐box” approach with the computation/data‐driven (feature fusion) “black‐box” approach. This will allow us to utilize data features that are educationally and contextually meaningful. This paper aims to extend current methodological paradigms, and puts into practice the proposed approach in an adaptive self‐assessment case study taking advantage of new, cutting‐edge, interdisciplinary work on building pipelines for educational data, using innovative tools and techniques. What is already known about this topic Capturing and measuring learners' engagement and behavior using physiological data has been explored during the last years and exhibits great potential. Effortless behavioral patterns commonly exhibited by learners, such as “cheating,” “guessing” or “gaming the system” counterfeit the learning outcome. Multimodal data can accurately predict learning engagement, performance and processes. What this paper adds Generalizes a methodology for building machine learning pipelines for multimodal educational data, using a modularized approach, namely the “grey‐box” approach. Showcases that fusion of eye‐tracking, facial expressions and arousal data provide the best prediction of effort and performance in adaptive learning settings. Highlights the importance of fusing data from different channels to obtain the most suited combinations from the different multimodal data streams, to predict and explain effort and performance in terms of pervasiveness, mobility and ubiquity. Implications for practice and/or policy Learning analytics researchers shall be able to use an innovative methodological approach, namely the “grey‐box,” to build machine learning pipelines from multimodal data, taking advantage of artificial intelligence capabilities in any educational context. Learning design professionals shall have the opportunity to fuse specific features of the multimodal data to drive the interpretation of learning outcomes in terms of physiological learner states. The constraints from th...

show abstract

Section: Motivation Of the Research And Research Questionmentioning

confidence: 99%

Section: Related Work: Utilizing Multimodal Data To Predict Learning mentioning

confidence: 99%

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Sharma¹,

Papamitsiou²,

Giannakos³

2019

Brit J Educational Tech

View full text Add to dashboard Cite

show abstract

“…These contexts vary from games [18], [25], to assessment systems [40], to adaptive systems [32], to collaborative systems [24]. However, one common factor in these studies is the use of multiple data streams (e.g., gaze, facial expressions, Electroencephalography (EEG), heart rate, log data) to predict and explain learning performance [18], [40], behaviour [3], [42] or experience [37], [38].…”

Section: B Multi-modal Data-based Predictions In Educationmentioning

confidence: 99%

Motion-Based Educational Games: Using Multi-Modal Data to Predict Player’s Performance

Lee-Cultura

Sharma

Papavlasopoulou

et al. 2020

2020 IEEE Conference on Games (CoG)

View full text Add to dashboard Cite

Multi-Modal Data (MMD) can help educational games researchers understand the synergistic relationship between player's movement and their learning experiences, and consequently uncover insights that may lead to improved design of movement-based game technologies for learning. Predicting player performance fosters opportunities to cultivate heightened educational experiences and outcomes. However, predicting player's performance utilising player-generated MMD during their interactions with educational Motion-Based Touchless Games (MBTG) is challenging. To bridge this gap, we implemented an in-situ study where 26 users, age 11, played 2 maths MBTGs in a single 20-30 minute session. We collected player's MMD (i.e., gaze data from eye-tracking glasses, physiological data from wristbands, and skeleton data from Kinect) produced during game-play. To investigate the potential of MMD for predicting player's academic performance, we used machine learning techniques and MMD derived from player's game-play. This allowed us to identify the MMD features that drive rapid highly accurate predictions of players' academic performance in educational MBTGs. This might allow us to provide realtime proactive feedback to the player to support them through their educational gaming experience. Our analysis compared two data lengths corresponding to half and full duration of the player's question solving time. We showed that all combinations of extracted features associated with gaze, physiological, and skeleton data, predicted student performance more accurately than the majority baseline. Additionally, the most accurate prediction of player's performance derived from the combination of gaze and physiological data for both full and half data lengths. Our findings emphasise the significance of using MMD for realtime performance prediction in educational MBTG and offer implications for practice.

show abstract

“…Additional publications relevant to the study questions we're engaged in were retrieved using Google Scholar and Google. The systematic reviews' references [17,18,23,36,[42][43][44][45][46][47][48][49][50] were also examined to ensure that no relevant papers were neglected. Only Englishlanguage articles from conference proceedings, transactions, magazines, books, essays, technical reports, white papers, and manufacturer's technical guides were studied.…”

Section: Methodsmentioning

confidence: 99%

Towards Collaborative and Intelligent Learning Environments Based on Eye Tracking Data and Learning Analytics: A Survey

Wang

Harter

2021

IEEE Access

View full text Add to dashboard Cite

The current pandemic has significantly impacted educational practices, modifying many aspects of how and when we learn. In particular, remote learning and the use of digital platforms have greatly increased in importance. Online teaching and e-learning provide many benefits for information retention and schedule flexibility in our on-demand world while breaking down barriers caused by geographic location, physical facilities, transportation issues, or physical impediments. However, educators and researchers have noticed that students face a learning and performance decline as a result of this sudden shift to online teaching and e-learning from classrooms around the world. In this paper, we focus on reviewing eyetracking techniques and systems, data collection and management methods, datasets, and multi-modal learning data analytics for promoting pervasive and proactive learning in educational environments. We then describe and discuss the crucial challenges and open issues of current learning environments and data learning methods. The review and discussion show the potential of transforming traditional ways of teaching and learning in the classroom, and the feasibility of adaptively driving learning processes using eye-tracking, data science, multimodal learning analytics, and artificial intelligence. These findings call for further attention and research on collaborative and intelligent learning systems, plug-and-play devices and software modules, data science, and learning analytics methods for promoting the evolution of face-to-face learning and e-learning environments and enhancing student collaboration, engagement, and success.

show abstract

Classroom Social Signal Analysis

Cited by 11 publications

References 4 publications

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Motion-Based Educational Games: Using Multi-Modal Data to Predict Player’s Performance

Towards Collaborative and Intelligent Learning Environments Based on Eye Tracking Data and Learning Analytics: A Survey

Contact Info

Product

Resources

About