3D Morphable Face Models and Their Applications

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Awais

et al. 2018

Self Cite

358

212

We present a new loss function, namely Wing loss, for robust facial landmark localisation with Convolutional Neural Networks (CNNs). We first compare and analyse different loss functions including L2, L1 and smooth L1. The analysis of these loss functions suggests that, for the training of a CNN-based localisation model, more attention should be paid to small and medium range errors. To this end, we design a piece-wise loss function. The new loss amplifies the impact of errors from the interval (-w, w) by switching from L1 loss to a modified logarithm function.To address the problem of under-representation of samples with large out-of-plane head rotations in the training set, we propose a simple but effective boosting strategy, referred to as pose-based data balancing. In particular, we deal with the data imbalance problem by duplicating the minority training samples and perturbing them by injecting random image rotation, bounding box translation and other data augmentation approaches. Last, the proposed approach is extended to create a two-stage framework for robust facial landmark localisation. The experimental results obtained on AFLW and 300W demonstrate the merits of the Wing loss function, and prove the superiority of the proposed method over the state-of-the-art approaches.

Section: Introductionmentioning

confidence: 99%

Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Awais

et al. 2018

Self Cite

358

212

“…However, the laborious work of manually annotating facial landmarks for face images is tedious. One alternative is to synthesise virtual training samples using a generative model, such as the 3D morphable face model [25], [53], [67]. However, the collection of 3D face scans and the construction of a 3D face model are very involved compared with the data collection and model construction of a 2D face model.…”

Section: The Use Of Ut-aam In Facial Landmark Detectionmentioning

confidence: 99%

A Unified Tensor-based Active Appearance Model

ACM Trans. Multimedia Comput. Commun. Appl.

Christmas

et al. 2019

Self Cite

Appearance variations result in many difficulties in face image analysis. To deal with this challenge, we present a Unified Tensor-based Active Appearance Model (UT-AAM) for jointly modelling the geometry and texture information of 2D faces. For each type of face information, namely shape and texture, we construct a unified tensor model capturing all relevant appearance variations. This contrasts with the variationspecific models of the classical tensor AAM. To achieve the unification across pose variations, a strategy for dealing with selfoccluded faces is proposed to obtain consistent shape and texture representations of pose-varied faces. In addition, our UT-AAM is capable of constructing the model from an incomplete training dataset, using tensor completion methods. Last, we use an effective cascaded-regression-based method for UT-AAM fitting. With these advancements, the utility of UT-AAM in practice is considerably enhanced. As an example, we demonstrate the improvements in training facial landmark detectors through the use of UT-AAM to synthesise a large number of virtual samples. Experimental results obtained using the Multi-PIE and 300-W face datasets demonstrate the merits of the proposed approach.

“…Owing to these advantages, 3DMM has been widely used in many areas including, but 115 not limited to, pattern recognition [3,12,22,26,52,57]. For an overview of 3DMM's applications the reader is referred to [30].…”

Section: Related Workmentioning

confidence: 99%

“…Thus for each subject it provides information equivalent to hundreds of images of different poses. In addition, by physically separating the face model from an illumination model, it can also generate an 30 for instance, two cohorts, namely Caucasian and Chinese faces. The shapes of these two groups are clearly different.…”

Section: Introductionmentioning

confidence: 99%

Gaussian mixture 3D morphable face model

Koppen

et al. 2018

Pattern Recognition

Self Cite

Abstract3D Morphable Face Models (3DMM) have been used in pattern recognition for some time now. They have been applied as a basis for 3D face recognition, as well as in an assistive role for 2D face recognition to perform geometric and photometric normalisation of the input image, or in 2D face recognition system training. The statistical distribution underlying 3DMM is Gaussian. However, the single-Gaussian model seems at odds with reality when we consider different cohorts of data, e.g. Black and Chinese faces. Their means are clearly different. This paper introduces the Gaussian Mixture 3DMM (GM-3DMM) which models the global population as a mixture of Gaussian subpopulations, each with its own mean. The proposed GM-3DMM extends the traditional 3DMM naturally, by adopting a shared covariance structure to mitigate small sample estimation problems associated with data in high dimensional spaces. We construct a GM-3DMM, the training of which involves a multiple cohort dataset, SURREY-JNU, comprising 942 3D face scans of people with mixed backgrounds. Experiments in fitting the GM-3DMM to 2D face images to facilitate their geometric and photometric normalisation for pose and illumination invariant face recognition demonstrate the merits of the proposed mixture of Gaussians 3D face model.