Single image-based head pose estimation with spherical parametrization and 3D morphing

Yuan, Hui; Li, Mengyu; Hou, Junhui; Xiao, Jimin

doi:10.1016/j.patcog.2020.107316

Cited by 28 publications

(10 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, it would be interesting to explore the extension of the proposed framework to perform 3D pose estimation as part of our future research. In accordance with recent studies, 3D pose projection from 2D images can be achieved, either by employing geometric relationships between 2D keypoint positions and 3D human pose models [58], or by leveraging occlusion-robust pose-maps (ORPM) in combination with annotated 3D poses [3,31].…”

Section: Avenues For Further Researchsupporting

confidence: 53%

EfficientPose: Scalable single-person pose estimation

2020

View full text Add to dashboard Cite

Single-person human pose estimation facilitates markerless movement analysis in sports, as well as in clinical applications. Still, state-of-the-art models for human pose estimation generally do not meet the requirements of real-life applications. The proliferation of deep learning techniques has resulted in the development of many advanced approaches. However, with the progresses in the field, more complex and inefficient models have also been introduced, which have caused tremendous increases in computational demands. To cope with these complexity and inefficiency challenges, we propose a novel convolutional neural network architecture, called EfficientPose, which exploits recently proposed EfficientNets in order to deliver efficient and scalable single-person pose estimation. EfficientPose is a family of models harnessing an effective multi-scale feature extractor and computationally efficient detection blocks using mobile inverted bottleneck convolutions, while at the same time ensuring that the precision of the pose configurations is still improved. Due to its low complexity and efficiency, EfficientPose enables real-world applications on edge devices by limiting the memory footprint and computational cost. The results from our experiments, using the challenging MPII single-person benchmark, show that the proposed EfficientPose models substantially outperform the widely-used OpenPose model both in terms of accuracy and computational efficiency. In particular, our top-performing model achieves state-of-the-art accuracy on single-person MPII, with low-complexity ConvNets.

show abstract

Section: Avenues For Further Researchsupporting

confidence: 53%

EfficientPose: Scalable single-person pose estimation

2020

View full text Add to dashboard Cite

show abstract

“…All the above methods used the depth cue of the facial image. In recent work, Yuan et al [19] proposed a 3D morphing method with spherical parameterization which will deform an existing 3D facial model with the help of four non-coplanar 2D facial feature point along with all the three directions of yaw, pitch and roll.…”

Section: ) Learning From Geometrymentioning

confidence: 99%

Learning 3D Head Pose From Synthetic Data: A Semi-Supervised Approach

et al. 2021

View full text Add to dashboard Cite

Accurate head pose estimation from 2D image data is an essential component of applications such as driver monitoring systems, virtual reality technology, and human-computer interaction. It enables a better determination of user engagement and attentiveness. The most accurate head pose estimators are based on Deep Neural Networks that are trained with the supervised approach and rely primarily on the accuracy of training data. The acquisition of real head pose data with a wide variation of yaw, pitch and roll is a challenging task. Publicly available head pose datasets have limitations with respect to size, resolution, annotation accuracy and diversity. In this work, a methodology is proposed to generate pixel-perfect synthetic 2D headshot images rendered from high-quality 3D synthetic facial models with accurate head pose annotations. A diverse range of variations in age, race, and gender are also provided. The resulting dataset includes more than 300k pairs of RGB images with corresponding head pose annotations. A wide range of variations in pose, illumination and background are included. The dataset is evaluated by training a state-of-the-art head pose estimation model and testing against the popular evaluation-dataset Biwi. The results show that training with purely synthetic data generated using the proposed methodology achieves close to state-of-the-art results on head pose estimation which are originally trained on real human facial datasets. As there is a domain gap between the synthetic images and real-world images in the feature space, initial experimental results fall short of the current state-of-the-art. To reduce the domain gap, a semisupervised visual domain adaptation approach is proposed, which simultaneously trains with the labelled synthetic data and the unlabeled real data. When domain adaptation is applied, a significant improvement in model performance is achieved. Additionally, by applying a data fusion-based transfer learning approach, better results are achieved than previously published work on this topic.INDEX TERMS Head pose estimation, synthetic face, face dataset, visual domain adaptation.

show abstract

“…In computer vision, the pose of an object refers to its position relative to the camera's position. To calculate the position information of the face in the 3D space, the feature points of the face are detected on the image in the 2D space, and the position information of these feature points is extracted; then get the head rotation matrix is got by matching the feature points of 2D face with those of the 3D face model, and finally the Euler angle of the head is calculated according to the rotation matrix; the aforementioned steps are a usual way to estimate head poses [ 53 – 57 ]. The attitude of an object relative to the camera can be represented by a rotation matrix and a translation matrix; the translation matrix (expressed by T in ( 3 )) is the spatial position relation matrix of the object relative to the camera, and the rotation matrix (expressed by R in ( 3 )) is the spatial attitude matrix of the object relative to the camera.…”

Section: Proposal and Design Of Our Approachmentioning

confidence: 99%

Learning State Assessment in Online Education Based on Multiple Facial Features Detection

Cui

Cao

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Considering that most of online training is not effectively supervised, this article presents an online leaning state assessment approach which combines blink detection, yawn detection, and head pose estimation. Blink detection is realized by computing the eye aspect ratio and the ratio of closed eye frames to the total frames per unit time to evaluate the degree of eye fatigue. Yawn detection is implemented by computing the aspect ratio of the mouth by using the feature points of the inner lip and combining it with the time of opening mouth to distinguish the mouth state. Head pose estimation is first implemented by calculating the head rotation matrix by matching the feature points of 2D face with the 3D face model and then calculating the Euler angle of the head according to the rotation matrix to evaluate the change of the head pose. Especially in yawn detection, we employ the feature points of inner lips in the calculation of the mouth aspect ratio to avoid the impact of lip thickness of various participants. Furthermore, the blink detection, yawn detection, and head pose estimation are first calculated based on the two-dimensional grayscale image of human face, which could reduce the computational complexity and improve the real-time performance of detection. Finally, combining the values of blinking, yawning, and head pose, multiple groups of experiments are carried out to assess the state of different online learners; then, the learning state is evaluated by analyzing the numerical changes of the three characteristics. Experimental results show that our approach could effectively evaluate the state of online learning and provide support for the development of online education.

show abstract

Single image-based head pose estimation with spherical parametrization and 3D morphing

Cited by 28 publications

References 54 publications

EfficientPose: Scalable single-person pose estimation

EfficientPose: Scalable single-person pose estimation

Learning 3D Head Pose From Synthetic Data: A Semi-Supervised Approach

Learning State Assessment in Online Education Based on Multiple Facial Features Detection

Contact Info

Product

Resources

About