Behaviour detection in crowded classroom scenes via enhancing features robust to scale and perspective variations

Abstract

Detecting human behaviours in images of crowded classroom scenes is a challenging task, due to the large variations of humans in scale and pose perspective. In this paper, two modules are proposed to tackle these two variations. First, an attention‐based RoI (region‐of‐interest) extractor is designed to handle scale variation. Feature fusion and attention mechanism are used to improve the RoI feature with more local and global information. Second, a transformation‐based detection head is introduced to handle perspective variation. The spatial transformation is adopted to extract consistent representation under various perspectives. Moreover, since there is a lack of proper datasets for human behaviour detection in classroom scenes, a new dataset is created, namely CLBD. The experiments on the proposed dataset demonstrate that the modules obtain significant improvements of performance over the state‐of‐the‐art detectors.

Research on Students’ Action Behavior Recognition Method Based on Classroom Time-Series Images

“…Banerjee et al [25] proposed an improved SSD object detection model to recognize the behavior of students and teachers in the teaching laboratory. Liu et al [26] improved the two-stage object detection network and proposed a new ROI extractor SAA module and a new detection head RST module for student behavior detection in classroom scenes. Huang et al [27] constructed a deep neural network to extract facial keypoints, recognize students' head postures and expressions, and classify classroom behaviors by combining the head gestures and expressions.…”