ET-YOLOv5s: Toward Deep Identification of Students’ in-Class Behaviors

Abstract

Recognizing students' behaviors in classes based on videos of their activity plays an important role in improving teaching quality and paying attention to the healthy growth of students. The existing student behavior recognition methods mainly focus on the behavior of the single student and have low performance and efficiency. To recognize the behaviors of multiple students in the classroom at the same time, we propose a fast and effective solution, called ET-YOLOv5s, which is an improved YOLOv5s with ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) and a tiny object detection module in this paper. First, the ESRGAN is adopted to generate high-definition images from original images of real classroom environments in colleges to improve the recognition accuracy. Then a tiny object detection module is added to the YOLOv5s to detect the smaller objects on high-definition images, such as phones, books, and students sitting at the back of the classroom. Finally, the experimental results show that the proposed solution can detect 11 kinds of students' behaviors in classrooms, including playing mobile phones, having class, sitting with hands on face, sitting turn right, sitting turn left, bowing, sleeping, and so on. Compared with YOLOv4, YOLOv5s, and other tiny target detection algorithms, it has better detection performance and can effectively deal with the behavior recognition of multiple students.

A model for new media data mining and analysis in online English teaching using long short-term memory (LSTM) network

“…Through the analysis and verification of teaching video cases of this method, the differences of teaching modes of various disciplines can be obtained by cluster analysis for a specific discipline ( Zhou et al, 2022 ). Li et al (2022) proposed a class behavior recognition algorithm that integrates human body pose estimation and target detection to extract and detect students’ skeleton, providing data support for optimizing teaching design and implementing teaching intervention ( Zhao, Zhu & Niu, 2023 ). At present, this kind of application can only detect students’ behaviors, obtain and analyze the classroom behavior data of most students in the data, and lack the classroom behavior evaluation for a certain student, and the data cannot be used as the evaluation basis for all students in the classroom.…”

Research on lightweight pedestrian detection based on improved YOLOv5

“…To achieve a lightweight model and make it easier to port to edge devices, YOLOv5s is chosen as the base model in this paper. The following is a schematic diagram of the overall network structure of YOLOv5s [9].…”

An Improved YOLOv7-CO Classroom Behavior Detection Algorithm Combining Coordinate Attention Mechanism and Omni-Dimensional Dynamic Convolution

“…Zheng et al [9] based on a two-stage network Faster R-CNN framework to select candidate frames before classifying and localizing student behavior, successfully and accurately identified three behaviors: raising hands, standing and sleeping. Li Lina et al [10] et al directly output predictions based on an improved onestage YOLOv5s network and successfully recognized four behaviors: listening, looking down, lying down and standing up.…”