A survey on automatic engagement recognition methods: online and traditional classroom

Abstract

Student engagement in a learning environment is directly related to students’ perception and involvement of the educational activities in the class, along with their physical and mental health. This paper presents an extensive survey of the various automatic engagement detection approaches and algorithms based on computer vision, physiological and neurological signals analysis-based methods. The computer vision-based techniques depend on the traits captured by image sensors such as facial expressions, gesture and posture analysis, and gaze direction. The physiological and neurological signal based approach depends on the sensor data, like heart rate (HR), electroencephalogram (EEG), blood pressure (BP), and galvanic skin response (GSR). A brief analysis of the available datasets for Engagement Recognition and its features are also summarized. This study highlights a few commercially available wearables which provides the physiological signals that helps in student’s attentivity recognition. Our study reveal that the accuracy of engagement recognition system will increase if we increase the number of modalities used. In this survey, we intend to support the upcoming researchers as well as tutors of smart education set up by providing an overview of existing or proposed approaches of automatic engagement detection techniques in different scenarios.

Research on Learning Concentration Recognition with Multi-Modal Features in Virtual Reality Environments

“…As shown in Figure 5, vision data had better a F1 score compared to interaction data. This finding aligns with previous research conducted in online learning environments [55,56], STEM environments [57], and traditional classroom environments [58], where vision data captured through cameras have shown high effectiveness in recognizing concentration. Facial expressions, eye gaze, and other related data serve as important sources for recognizing concentration levels [55][56][57][58].…”

“…This finding aligns with previous research conducted in online learning environments [55,56], STEM environments [57], and traditional classroom environments [58], where vision data captured through cameras have shown high effectiveness in recognizing concentration. Facial expressions, eye gaze, and other related data serve as important sources for recognizing concentration levels [55][56][57][58]. These results emphasized the continued significance of vision data as a valuable source for recognizing concentration in VR environments.…”

Multimodal Engagement Recognition From Image Traits Using Deep Learning Techniques

“…Many of the existing research for engagement recognition is based on the publicly available databases HBSC, Daisee and in the wild datasets. A short summary of publicly available datasets for facial emotion recognition and academic emotion of affective states along with its features are provided in the review articles [13], [33] Based on the review of the literature, it is evident that for determining the attention level of students, facial expression and gaze are critical modalities. The head pose has a substantial role in determining the gaze direction.…”

“…Physiological signals associated with engagement can be obtained from sensors such as Electroencephalogram (EEG), Electrocardiogram(ECG), Galvanic Skin Response(GSR) and Blood Pressure(BP). Sukumaran et al [13] conducted a review on various automatic attention tracking approaches and algorithms, both in online and traditional classes based on computer vision, physiological and neurological signals.…”

Exploring the Implementation of Multimedia Technology in Contemporary Home Product Design for Regional Culture Inheritance and Innovation

“…The texture should be delicate and realistic, while the colors should be chosen based on the room's intended use and visual effect. Additionally, decorative materials should be able to enhance indoor lighting, temperature, humidity, and provide sound insulation, heat insulation, fire prevention, and pollution prevention [15].…”