Holistic Analysis of the Classroom

Abstract

Should we judge the quality of the class by the grades the students and teacher get at the end of the semester or how the group collaborated during the semester towards acquiring new knowledge? Up until recently, the later approach was all too inaccessible due to complexity and time needed to evaluate every class. With the development of new technologies in different branches of video processing, gaze tracking and audio analysis we are getting the opportunity to go further with our analysis and go around the potential problem substitution into which we were previously forced.We present our efforts to record student-student and studentteacher interactions within a classroom eco-system. For this purpose, we developed a multi-camera system for observing teacher actions and students reactions throughout the class. We complemented the data with a mobile eye-tracker worn by the teacher, quantitative questionnaire data collection, as well as in-depth interviews with students about their impressions of the classes they took, and about our intervention. The seven-part experiment was conducted during the autumn semester of 2013, in two classes with over 60 participants.We present the conclusions we reached about the experiment format, visualize the preliminary results of our processing and discuss other options we are considering for our further experiments. We aim to explore further possibilities for analysing classroom life in order to create a more responsive environment to the needs of the students.

From signals to knowledge: A conceptual model for multimodal learning analytics

“…The movements of the torso can provide GBM, which is typically derived from video cameras. GBM was used by Raca and Dillenbourg () in their study for assessing students' attention from their body posture, gesturing, and other cues. Similarly, Bosch et al () used GBM to detect learners' emotions in combination with facial expression and learning activity.…”

“…Facial expressions are highly investigated in learning for emotion recognition in the affective computing research and have been quite extensively used in multimodal human–computer interaction experiments (e.g., Alyuz et al, ; Bosch et al, ; Hussain et al, ). Eye‐tracking is commonly used as an indicator for learners' attention has also been used with multimodal data sets (Edwards et al, ; Prieto, Sharma, Dillenbourg, & Rodríguez‐Triana, ; Raca & Dillenbourg, ). Finally, an analysis of the speech spans from paralanguage analysis like speaking time, keywords pronounced, or prosodic features like tone and pitch (e.g., Prieto et al, ) to actual recognition of spoken words in dialogic settings like student–teacher interactions (D'mello et al, ).…”

“…The movements of the torso can provide GBM, which is typically derived from video cameras. GBM was used by Raca and Dillenbourg (2014) The motoric modalities of the head include analysis of the facial expressions, eye movements, and speech analysis. These three body parts can provide relevant information to the point that three wellestablished research communities are dedicated to advancing the techniques and methodologies for data acquisition.…”

“…Facial expressions are highly investigated in learning for emotion recognition in the affective computing research and have been quite extensively used in multimodal human-computer interaction experiments (e.g., Alyuz et al, 2016;Bosch et al, 2015;Hussain et al, 2012). Eye-tracking is commonly used as an indicator for learners' attention has also been used with multimodal data sets (Edwards et al, 2017;Prieto, Sharma, Dillenbourg, & Rodríguez-Triana, 2016;Raca & Dillenbourg, 2014).…”

Read Between the Lines

“…Sensors can monitor both the physical environment where the learner operates and the learner's behaviour including 360-degrees body movements, physiological responses such as heart rate or body temperature, or interpersonal communication, student-teacher or student-peers discussions. In related literature, sensors have been used to track different modalities such as hand gestures [24,40], gross body movements [7], eyetracking [28,29], facial expressions [3,4]. Sensors were also used to monitor physiological signals such as heart rate [1,16], galvanic skin response [14,25], brain waves [1,28].…”