Joint Multi-View Face Alignment in the Wild

Abstract

The de facto algorithm for facial landmark estimation involves running a face detector with a subsequent deformable model fitting on the bounding box. This encompasses two basic problems: i) the detection and deformable fitting steps are performed independently, while the detector might not provide best-suited initialisation for the fitting step, ii) the face appearance varies hugely across different poses, which makes the deformable face fitting very challenging and thus distinct models have to be used (e.g., one for profile and one for frontal faces). In this work, we propose the first, to the best of our knowledge, joint multi-view convolutional network to handle large pose variations across faces in-the-wild, and elegantly bridge face detection and facial landmark localisation tasks. Existing joint face detection and landmark localisation methods focus only on a very small set of landmarks. By contrast, our method can detect and align a large number of landmarks for semi-frontal (68 landmarks) and profile (39 landmarks) faces. We evaluate our model on a plethora of datasets including standard static image datasets such as IBUG, 300W, COFW, and the latest Menpo Benchmark for both semi-frontal and profile faces. Significant improvement over state-of-the-art methods on deformable face tracking is witnessed on 300VW benchmark. We also demonstrate state-ofthe-art results for face detection on FDDB and MALF datasets.Index Terms-Joint multi-view face alignment, Cascade face detection

DeCaFA: Deep Convolutional Cascade for Face Alignment in the Wild

“…Our [26] 49.87 5.08 Densereg+MDM [1] 52. 19 3.67 JMFA [3] 54.9 1.00 JMFA-MENPO [3] 60.7 0.33 LAB [19] 58.9 0.83 DeCaFA 0.661 0.15 [22] 4.35 CFSS [26] 3,95 DSRN [13] 3.08 AAN [23] 2.99 DeCaFA 2.10 approach is the best by a significant margin. Noteworthy, even though we use auxiliary data from 300W and WFLW, we do not use data from the val partition of CelebA, contrary to [13,23], thus there is significant room for improvement.…”

“…Note that DeCaFA trained only on 300W trainset has a ME of 3.69% and is already very competitive with recent approaches [9,5,4,8], thanks to its end-to-end cascade architecture. DeCaFA is competitive with the best approaches, LAB [19] and DAN-MENPO [8] as well as JMFA-MENPO [3], which also use external data. Table 2 shows a comparison between our method and LAB [19] on WFLW database.…”

Semantic Alignment: Finding Semantically Consistent Ground-Truth for Facial Landmark Detection

“…To further improve the performance, many popular semantic segmentation and human pose estimation frameworks are used for face alignment [29,5,2,14]. For each landmark, they predict a heatmap which contains the probability of the corresponding landmark.…”

Aggregation via Separation: Boosting Facial Landmark Detector With Semi-Supervised Style Translation

“…Recently, with the power of deep neural networks, regression-based models are able to produce better results. They are mainly divided into two streams of direct coordinate regression [80,45,63,50] and heatmap-based regression [51,5,13,75,48]. Meanwhile, in [80], auxiliary attributes were used to learn a discriminative representation.…”