ICAPD Framework and simAM-YOLOv8n for Student Cognitive Engagement Detection in Classroom

Abstract

Research has shown that cognitive engagement plays a key role in effective learning, resulting in extensive efforts have been devoted to measuring it. Whereas most of the literature explores manual methods to measure cognitive engagement, the research on automatic detection of cognitive engagement levels in real classrooms is very limited. Automatic detection of cognitive engagement has been a problem for a long time due to the lack of behavior annotation guidance and effective detection algorithms. For the first challenge, a theory of cognitive engagement called Interactive-Constructive-Active-Passive-Disengage (ICAPD) was proposed in this paper. ICAPD links visual behaviors with cognitive engagement in the classroom. According to the ICAPD framework, a cognitive engagement dataset was constructed to train the detection model. To tackle the second challenge, the simAM-based You Only Look Once version 8 Nano (simAM-YOLOv8n) model, which uses simAM attention module to strengthen feature extraction, was designed to detect different levels of cognitive engagement precisely and efficiently. The experimental results on the self-build dataset have demonstrated the effectiveness of the proposed theory framework and detection algorithm, indicating that the proposed methodology could be used to detect real-time cognitive engagement in the classroom scenario. This work has the potential to help teachers to carry out learning analysis and instructional adjustments.

Occlusion Robust Cognitive Engagement Detection in Real-World Classroom

“…We further defined different degrees of cognitive engagement in university classrooms, referring to the work of [ 9 ]. The defined degrees are presented in Table 2 .…”

“…As shown in Figure 5, the generated images were annotated using the LabelImg data annotation tool. Moreover, based on the ICAPD framework [9], the locations and behaviors of the individual students were precisely marked with bounding boxes (seen in Figure 6). We further defined different degrees of cognitive engagement in university classrooms, referring to the work of [9].…”

“…Moreover, based on the ICAPD framework [9], the locations and behaviors of the individual students were precisely marked with bounding boxes (seen in Figure 6). We further defined different degrees of cognitive engagement in university classrooms, referring to the work of [9]. The defined degrees are presented in Table 2.…”

“…Therefore, visual data are suitable for detecting cognitive engagement. The ICAPD dataset [ 9 ] is an example of an image-based cognitive engagement dataset, but its small scale makes it challenging to ensure the robustness of models. Target detection technology (e.g., YOLO [ 18 ]) is a popular automatic assessment method for students and their behaviors.…”

“…[8]). When considering the ICAP framework, YOLO applied to cognitive engagement detection offers three advantages [2,9]: (1) using low-cost visual data, (2) tracking the location of students' behaviors, (3) efficiently establishing the relationship between behaviors and cognitive engagement, and (4) measuring students' cognitive levels.…”