Student Behavior Recognition From Heterogeneous View Perception in Class Based on 3-D Multiscale Residual Dense Network for the Analysis of Case Teaching

Liu, Hui; Liu, Yang; Zhang, Ran; Wu, Xia

doi:10.3389/fnbot.2021.675827

Cited by 9 publications

(6 citation statements)

References 38 publications

(40 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the study based on videos, Liu et al [29] proposed a 3D multi-scale residual dense network based on heterogeneous view perception for recognizing students' classroom behaviors. Jisi et al [30] combined a spatial affine transform network with a convolutional neural network to extract the spatiotemporal features of the video and fused the spatiotemporal features to classify students' behaviors by using a weighted sum method.…”

Section: Behavior Recognition In Classroom Scenariosmentioning

confidence: 99%

Research on Students’ Action Behavior Recognition Method Based on Classroom Time-Series Images

Shou,

Yan,

Wen

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Students’ action behavior performance is an important part of classroom teaching evaluation. To detect the action behavior of students in classroom teaching videos, and based on the detection results, the action behavior sequence of individual students in the teaching time of knowledge points is obtained and analyzed. This paper proposes a method for recognizing students’ action behaviors based on classroom time-series images. First, we propose an improved Asynchronous Interaction Aggregation (AIA) network for student action behavior detection. By adding a Multi-scale Temporal Attention (MsTA) module and a Multi-scale Channel Spatial Attention (MsCSA) module to the fast pathway and slow pathway, respectively, the accuracy of student action behavior recognition is improved in SlowFast, which is the backbone network of the improved AIA network,. Second, the Equalized Focal Loss function is introduced to improve the category imbalance that exists in the student action behavior dataset. Experimental results on the student action behavior dataset show that the method proposed in this paper can detect different action behaviors of students in the classroom and has better detection performance compared to the original AIA network. Finally, based on the results of action behavior recognition, the seat number is used as the index to obtain the action behavior sequence of individual students during the teaching time of knowledge points and the performance of students in this period is analyzed.

show abstract

Section: Behavior Recognition In Classroom Scenariosmentioning

confidence: 99%

Research on Students’ Action Behavior Recognition Method Based on Classroom Time-Series Images

Shou,

Yan,

Wen

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Based on the spatiotemporal representation learning, Xie et al [16] presented a deep learning algorithm to evaluate the classroom poses of college students. Liu et al [17] proposed a 3-D multiscale residual dense network from the heterogeneous view perception for analysis of student behaviors in class. The above student behavior detection methods show impressive detection results.…”

Section: Introductionmentioning

confidence: 99%

“…As one of the typical algorithms of deep learning, the target detection algorithm has excellent object recognition performance and has become the preferred method of student behavior detection. Student behavior detection algorithms are mainly divided into two categories: behavior detection on images [19], [20] or videos [17]. For static images, the commonly used detection algorithms to classify students' behavior in two ways.…”

Section: Introductionmentioning

confidence: 99%

ET-YOLOv5s: Toward Deep Identification of Students’ in-Class Behaviors

Liu

Sun

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Recognizing students' behaviors in classes based on videos of their activity plays an important role in improving teaching quality and paying attention to the healthy growth of students. The existing student behavior recognition methods mainly focus on the behavior of the single student and have low performance and efficiency. To recognize the behaviors of multiple students in the classroom at the same time, we propose a fast and effective solution, called ET-YOLOv5s, which is an improved YOLOv5s with ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) and a tiny object detection module in this paper. First, the ESRGAN is adopted to generate high-definition images from original images of real classroom environments in colleges to improve the recognition accuracy. Then a tiny object detection module is added to the YOLOv5s to detect the smaller objects on high-definition images, such as phones, books, and students sitting at the back of the classroom. Finally, the experimental results show that the proposed solution can detect 11 kinds of students' behaviors in classrooms, including playing mobile phones, having class, sitting with hands on face, sitting turn right, sitting turn left, bowing, sleeping, and so on. Compared with YOLOv4, YOLOv5s, and other tiny target detection algorithms, it has better detection performance and can effectively deal with the behavior recognition of multiple students.

show abstract

“…T HE performance of the student's learning behavior is crucial to instruction and assessment [1], [2]. Assessment of classroom learning behaviors usually includes manual and automated measures [3].…”

mentioning

confidence: 99%

“…However, classroom learning behavior recognition is a complex issue. The difficulties are (1) classroom learning behaviors are often obscured; (2) The learning behaviors for back-rowstudents are small objects. The object detection methods are not friendly to recognizing occluded objects and small objects in natural classrooms [11].…”

mentioning

confidence: 99%

YOLOv8n_BT: Research on Classroom Learning Behavior Recognition Algorithm Based on Improved YOLOv8n

Liu,

Jiang,

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Classroom learning behavior recognition can provide effective technical support for teaching and learning. However, in natural classroom teaching scenarios, classroom learning behaviors are often missed or falsely detected due to character occlusion and the small object. To tackle the above issues, this study proposed an improved classroom learning behavior recognition algorithm (YOLOv8n_BT) based on YOLOv8n. On the one hand, for the occlusion problem of classroom learning behaviors, this study incorporated the BRA into the Backbone to better capture feature information; on the other hand, for the small object problem of classroom learning behaviors for back-row-students, this study expanded a Tiny Object Detection Layer (TODL) to detect small targets better. Experiments show that the BRA and the TODL can significantly improve the model performance. The YOLOv8n_BT model, which incorporated both the BRA and the TODL into the YOLOv8n(baseline) model simultaneously, has the most significant performance improvement. Compared with the YOLOv8n(baseline), the YOLOv8n_BT model improved by 3.0%, 6.7%, 5.0%, 3.6%, and 9.0% on P, R, F1, mAP50, and mAP50-90, respectively. The detection performance of YOLOv8n_BT also outperforms other state-of-the-arts.

show abstract

Student Behavior Recognition From Heterogeneous View Perception in Class Based on 3-D Multiscale Residual Dense Network for the Analysis of Case Teaching

Cited by 9 publications

References 38 publications

Research on Students’ Action Behavior Recognition Method Based on Classroom Time-Series Images

Research on Students’ Action Behavior Recognition Method Based on Classroom Time-Series Images

ET-YOLOv5s: Toward Deep Identification of Students’ in-Class Behaviors

YOLOv8n_BT: Research on Classroom Learning Behavior Recognition Algorithm Based on Improved YOLOv8n

Contact Info

Product

Resources

About