Classroom Attention Analysis Based on Multiple Euler Angles Constraint and Head Pose Estimation

Xu, Xin; Teng, Xin

doi:10.1007/978-3-030-37731-1_27

Cited by 16 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Min [ 43 ] et al proposed a facial point positioning head pose estimation approach; the facial region could be properly identified by combining the AdaBoost algorithm with the skin color model; then the head position is estimated generally with Hoff circle method and the overall accuracy is proved by experiments. Xu and Teng [ 44 ] proposed a class grading module based on the conversion of Euler angles and attention of the head; the module takes the head Euler angle of three directions as input and uses spatial information for angle correction to obtain more accurate results. Churiwala et al [ 45 ] developed a system of drowsiness detection which mainly detects the four indicators: eye closing and blinking duration, blinking frequency, yawn, and head rotation; according to the calculation results, the fatigue degree of the driver is evaluated and audio signals are sent out when necessary to wake up the driver.…”

Section: Related Workmentioning

confidence: 99%

Learning State Assessment in Online Education Based on Multiple Facial Features Detection

Cui

Cao

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Considering that most of online training is not effectively supervised, this article presents an online leaning state assessment approach which combines blink detection, yawn detection, and head pose estimation. Blink detection is realized by computing the eye aspect ratio and the ratio of closed eye frames to the total frames per unit time to evaluate the degree of eye fatigue. Yawn detection is implemented by computing the aspect ratio of the mouth by using the feature points of the inner lip and combining it with the time of opening mouth to distinguish the mouth state. Head pose estimation is first implemented by calculating the head rotation matrix by matching the feature points of 2D face with the 3D face model and then calculating the Euler angle of the head according to the rotation matrix to evaluate the change of the head pose. Especially in yawn detection, we employ the feature points of inner lips in the calculation of the mouth aspect ratio to avoid the impact of lip thickness of various participants. Furthermore, the blink detection, yawn detection, and head pose estimation are first calculated based on the two-dimensional grayscale image of human face, which could reduce the computational complexity and improve the real-time performance of detection. Finally, combining the values of blinking, yawning, and head pose, multiple groups of experiments are carried out to assess the state of different online learners; then, the learning state is evaluated by analyzing the numerical changes of the three characteristics. Experimental results show that our approach could effectively evaluate the state of online learning and provide support for the development of online education.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning State Assessment in Online Education Based on Multiple Facial Features Detection

Cui

Cao

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Li [6] proposed a method that estimates the attention of 30 students in a class using a camera that real-time detects their head rotation without recognizing the eyeball pose; subsequently, they visualized the three states of student learning. Xu [11] proposed a multiple Euler angle constraint method to create a scoring module to analyze students' attention based on head pose estimation, where the system reported evaluates student attention levels from 0.0 to 1.0.…”

Section: Related Work 21 Head Pose Estimationmentioning

confidence: 99%

Online Classroom Evaluation System Based on Multi-Reaction Estimation

Peng,

Kikuchi,

Ozono

2021

Preprint

View full text Add to dashboard Cite

Online learning is more convenient than traditional face-to-face teaching methods. However, during real-time online classes, it is difficult for teachers to observe the reactions of all students simultaneously. Herein, we introduce an online education classroom evaluation system that enables teachers to adjust the speed of their lessons based on students' reactions. We aim to develop a method that can evaluate student participation based on multi-reaction of students. In this study, the system estimates the head poses and facial expressions of students through the camera and uses the information as criteria for assessing student participation. The estimated result enables the class quality to be categorized into positive, negative, and neutral, thereby allowing teachers to rearrange the class contents. Finally, we evaluate the performance of our system by testing the accuracy of student reaction estimation and our classroom evaluation method. CCS CONCEPTS• Applied computing → Computer-assisted instruction.

show abstract

“…Lin et al [ 7 ] designed a behavior recognition system that used the OpenPose framework [ 8 ] to obtain skeleton data, a correction scheme was proposed to reduce false connections, and then the skeleton data was modeled into different categories and action recognition was implemented using a classifier. Xu et al [ 9 ] proposed a method for head pose recognition based on Euler angles obtained from pose estimation. Yu et al [ 10 ] collected a classroom scenario dataset in a video game image called SynPose and employed a generative adversarial network (GAN) [ 11 ] to generate skeletons for transfer learning.…”

Section: Introductionmentioning

confidence: 99%

Pose Mask: A Model-Based Augmentation Method for 2D Pose Estimation in Classroom Scenes Using Surveillance Images

Liu

et al. 2022

Sensors

View full text Add to dashboard Cite

Solid developments have been seen in deep-learning-based pose estimation, but few works have explored performance in dense crowds, such as a classroom scene; furthermore, no specific knowledge is considered in the design of image augmentation for pose estimation. A masked autoencoder was shown to have a non-negligible capability in image reconstruction, where the masking mechanism that randomly drops patches forces the model to build unknown pixels from known pixels. Inspired by this self-supervised learning method, where the restoration of the feature loss induced by the mask is consistent with tackling the occlusion problem in classroom scenarios, we discovered that the transfer performance of the pre-trained weights could be used as a model-based augmentation to overcome the intractable occlusion in classroom pose estimation. In this study, we proposed a top-down pose estimation method that utilized the natural reconstruction capability of missing information of the MAE as an effective occluded image augmentation in a pose estimation task. The difference with the original MAE was that instead of using a 75% random mask ratio, we regarded the keypoint distribution probabilistic heatmap as a reference for masking, which we named Pose Mask. To test the performance of our method in heavily occluded classroom scenes, we collected a new dataset for pose estimation in classroom scenes named Class Pose and conducted many experiments, the results of which showed promising performance.

show abstract

Classroom Attention Analysis Based on Multiple Euler Angles Constraint and Head Pose Estimation

Cited by 16 publications

References 23 publications

Learning State Assessment in Online Education Based on Multiple Facial Features Detection

Learning State Assessment in Online Education Based on Multiple Facial Features Detection

Online Classroom Evaluation System Based on Multi-Reaction Estimation

Pose Mask: A Model-Based Augmentation Method for 2D Pose Estimation in Classroom Scenes Using Surveillance Images

Contact Info

Product

Resources

About