Learning State Assessment in Online Education Based on Multiple Facial Features Detection

Abstract

Considering that most of online training is not effectively supervised, this article presents an online leaning state assessment approach which combines blink detection, yawn detection, and head pose estimation. Blink detection is realized by computing the eye aspect ratio and the ratio of closed eye frames to the total frames per unit time to evaluate the degree of eye fatigue. Yawn detection is implemented by computing the aspect ratio of the mouth by using the feature points of the inner lip and combining it with the time of opening mouth to distinguish the mouth state. Head pose estimation is first implemented by calculating the head rotation matrix by matching the feature points of 2D face with the 3D face model and then calculating the Euler angle of the head according to the rotation matrix to evaluate the change of the head pose. Especially in yawn detection, we employ the feature points of inner lips in the calculation of the mouth aspect ratio to avoid the impact of lip thickness of various participants. Furthermore, the blink detection, yawn detection, and head pose estimation are first calculated based on the two-dimensional grayscale image of human face, which could reduce the computational complexity and improve the real-time performance of detection. Finally, combining the values of blinking, yawning, and head pose, multiple groups of experiments are carried out to assess the state of different online learners; then, the learning state is evaluated by analyzing the numerical changes of the three characteristics. Experimental results show that our approach could effectively evaluate the state of online learning and provide support for the development of online education.

Real-Time Drowsy Face Detection for Online Learning Based on Random Forest and Decision Tree Algorithms

“…If you look at Figure 1, the distance calculated is point 12 and point 16 as well as the distance between points P13 and P19, P15 and P17. The reason for choosing these points is that if you are sleepy or lack concentration, the student will open his mouth wide [18]. When the mouth is open, the distance between the points extending in the middle of the lips varies.…”

The Application of Adaptive Tolerance and Serialized Facial Feature Extraction to Automatic Attendance Systems

“…Each student, on enrolment, must provide a headshot which presents a complete set of facial features, as shown in Figure 3a. The convolutional neural network's (CNN) features, used within the context of the Maximum-Margin Object Detector (MMOD) (MMOD-CNN) face detector in the Dlib library (Dlib CNN), are adopted as the basis by which the faces and their positions within the object are detected, identified, and recognized [36][37][38]. The face within the object is extracted from the original headshot and converted to grayscale using OpenCV (Figure 3b).…”

Research on optimization algorithm of abnormal driving behavior detection based on PFLD

“…The negative pitch angle bias can be one of the characteristics of determining driver fatigue. The steps of solving the Euler angle of head pose estimation are as follows: (1) Obtaining 2D planar face keypoints based on PFLD (2) Performing face model matching in the 3D plane (3) Solving for the conversion relation between 2D plane coordinates and the corresponding 3D plane coordinates (4) Solving for the Euler angles by means of the resulting rotation matrix to obtain the Pitch values[18].…”