Building an automated engagement recognizer based on video analysis

Abstract

This paper presents a process to build a classifier in a datadriven way for recognizing engagement of children in a robotbased math quiz game. The process consists of collecting video recordings from HRI experiments; annotating the social signals and engagement states via video analysis; extracting feature vectors from the annotations and training classifiers. We conducted an experiment with 7 participants of 10 ∼ 11 years of age using an android robot EveR-4. With three coders annotating the video recordings and extracting features by snapshot model with 1-second time window, we achieved 84.83% recall performance.

Student behavior analysis to measure engagement levels in online learning environments

“…A smart online learning system is an innovative approach that can automatically adapt to learners' emotions, provide feedback about their motivations and practically incorporate new technologies and ways of expression by developing computer vision [1]. Engagement is one of the qualitative indicators in the learning process [2]. It has a three-dimensional structure during learning [3], namely behavioral engagement such as staying on task, emotional engagement such as being bored, and cognitive engagement such as focused attention [2].…”

“…Engagement is one of the qualitative indicators in the learning process [2]. It has a three-dimensional structure during learning [3], namely behavioral engagement such as staying on task, emotional engagement such as being bored, and cognitive engagement such as focused attention [2]. All these dimensions are important to measure engagement levels.…”

“…Among the various instruments that help measure student engagement are three popular tools: self-reports, observational checklists, and automated measurements [2]. Both self-reports and observational checklists are still very primitive and not suitable for real-time learning systems.…”

“…The affective computing domain has provided smart solutions to detect students' emotions. Some researchers highlight the importance of monitoring students' emotions and the way emotions can be detected as a reflection of student engagement [2]. Automated measurements are based on physiological and neurological sensor readings, like electroencephalography (EEG), heart rate, etc.…”

“…Smart e-learning systems based on student engagement have generated a great deal of interest among researchers of the computer vision community in order to advance research in the computer-based educational field. This is motivated by two major factors: (1) measuring student engagement plays a key role in building smart online learning systems (adaptation and motivation), so there is a need to develop an accurate automated system in order to improve the learning process, especially in a computer laboratory; (2) current approaches that rely only on emotions to detect engagement have low performance. They are not suitable for real-time systems as they use expensive tools and do not respect student privacy.…”