Cascade Multi-View Hourglass Model for Robust 3D Face Alignment

Deng, Jiankang; Zhou, Yuxiang; Cheng, Shiyang; Zaferiou, Stefanos

doi:10.1109/fg.2018.00064

Cited by 51 publications

(56 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We employ a normalization procedure based on [54], which is a revision of [45], but with a few small changes. We utilize state-of-the-art open-source implementations for face detection 2 [17] and facial landmarks detection 3 [5], respectively. We use the Surrey Face Model [20] as the reference 3D face model, and select 4 eye corners and 9 nose landmarks as described by the Multi-PIE 68-points markup [10] for PnP-based [25] head pose estimation.…”

Section: Appendixmentioning

confidence: 99%

Few-Shot Adaptive Gaze Estimation

Park

Mello²,

Molchanov³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

169

128

View full text Add to dashboard Cite

Inter-personal anatomical differences limit the accuracy of person-independent gaze estimation networks. Yet there is a need to lower gaze errors further to enable applications requiring higher quality. Further gains can be achieved by personalizing gaze networks, ideally with few calibration samples. However, over-parameterized neural networks are not amenable to learning from few examples as they can quickly over-fit. We embrace these challenges and propose a novel framework for Few-shot Adaptive GaZE Estimation (FAZE) for learning person-specific gaze networks with very few (≤ 9) calibration samples. FAZE learns a rotation-aware latent representation of gaze via a disentangling encoder-decoder architecture along with a highly adaptable gaze estimator trained using meta-learning. It is capable of adapting to any new person to yield significant performance gains with as few as 3 samples, yielding state-of-the-art performance of 3.18 • on GazeCapture, a 19% improvement over prior art.

show abstract

Section: Appendixmentioning

confidence: 99%

Few-Shot Adaptive Gaze Estimation

Park

Mello²,

Molchanov³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

169

128

View full text Add to dashboard Cite

show abstract

“…Since we organised the Menpo 3D competition, we could not submit an entry. However, as in the case of Menpo 2D competition, we have applied another our recent method (Deng et al 2018) for localising the 3DA-2D landmarks of the Menpo 3D Benchmark. This method extends our previous Fig.…”

Section: A New Strong Baseline For 3d Face Alignmentmentioning

confidence: 99%

The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking

et al. 2018

Self Cite

View full text Add to dashboard Cite

In this article, we present the Menpo 2D and Menpo 3D benchmarks, two new datasets for multi-pose 2D and 3D facial landmark localisation and tracking. In contrast to the previous benchmarks such as 300W and 300VW, the proposed benchmarks contain facial images in both semi-frontal and profile pose. We introduce an elaborate semi-automatic methodology for providing high-quality annotations for both the Menpo 2D and Menpo 3D benchmarks. In Menpo 2D benchmark, different visible landmark configurations are designed for semi-frontal and profile faces, thus making the 2D face alignment full-pose. In Menpo 3D benchmark, a united landmark configuration is designed for both semi-frontal and profile faces based on the correspondence with a 3D face model, thus making face alignment not only full-pose but also corresponding to the realworld 3D space. Based on the considerable number of annotated images, we organised Menpo 2D Challenge and Menpo 3D Challenge for face alignment under large pose variations in conjunction with CVPR 2017 and ICCV 2017, respectively. The results of these challenges demonstrate that recent deep learning architectures, when trained with the abundant data, lead to excellent results. We also provide a very simple, yet effective solution, named Cascade Multi-view Hourglass Model, to 2D and 3D face alignment. In our method, we take advantage of all 2D and 3D facial landmark annotations in a joint way. We not only capitalise on the correspondences between the semi-frontal and profile 2D facial landmarks but also employ joint supervision from both 2D and 3D facial landmarks. Finally, we discuss future directions on the topic of face alignment. Keywords 2D face alignment • 3D face alignment • Menpo challenge 1 Introduction Facial landmark localisation and tracking on images and videos captured in unconstrained recording conditions is a problem that has received a lot of attention during the past few years. This is attributed to the fact that it is a neces

show abstract

“…The first step to edit an image is to locate landmark points that will be used for fitting the 3DMM. We first perform face detection with the face detection model from [73] and then utilize [18] to localize 68 2D facial landmark points which are aware of the 3D structure of the face, in the sense that points on occluded parts of the face (most commonly part of the jawline) are correctly localized.…”

Section: Face Detection and Landmark Localizationmentioning

confidence: 99%

Deep Neural Network Augmentation: Generating Faces for Affect Analysis

et al. 2020

Self Cite

View full text Add to dashboard Cite

This paper presents a novel approach for synthesizing facial affect; either in terms of the six basic expressions (i.e., anger, disgust, fear, joy, sadness and surprise), or in terms of valence (i.e., how positive or negative is an emotion) and arousal (i.e., power of the emotion activation). The proposed approach accepts the following inputs: i) a neutral 2D image of a person; ii) a basic facial expression or a pair of valencearousal (VA) emotional state descriptors to be generated, or a path of affect in the 2D VA Space to be generated as an image sequence. In order to synthesize affect in terms of VA, for this person, 600, 000 frames from the 4DFAB database were annotated. The affect synthesis is implemented by fitting a 3D Morphable Model on the neutral image, then deforming the reconstructed face and adding the inputted affect, and blending the new face with the given affect into the original image. Qualitative experiments illustrate the generation of realistic images, when the neutral image is sampled from thirteen well known lab-controlled or in-the-wild databases, including Aff-Wild, AffectNet, RAF-DB; comparisons with Generative Adversarial Networks (GANs) show the higher quality achieved by the proposed approach. Then, quantitative experiments are conducted, in which the synthesized images are used for data augmentation in training Deep Neural Networks to perform affect recognition over all databases; greatly improved performances are achieved when compared with state-of-theart methods, as well as with GAN-based data augmentation, in all cases.

show abstract

Cascade Multi-View Hourglass Model for Robust 3D Face Alignment

Cited by 51 publications

References 39 publications

Few-Shot Adaptive Gaze Estimation

Few-Shot Adaptive Gaze Estimation

The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking

Deep Neural Network Augmentation: Generating Faces for Affect Analysis

Contact Info

Product

Resources

About