Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade

Abstract

We present a new approach to localize extensive facial landmarks with a coarse-to-fine convolutional network cascade. Deep convolutional neural networks (DCNN) have been successfully utilized in facial landmark localization for two-fold advantages: 1) geometric constraints among facial points are implicitly utilized; 2) huge amount of training data can be leveraged. However, in the task of extensive facial landmark localization, a large number of facial landmarks (more than 50 points) are required to be located in a unified system, which poses great difficulty in the structure design and training process of traditional convolutional networks. In this paper, we design a four-level convolutional network cascade, which tackles the problem in a coarse-to-fine manner. In our system, each network level is trained to locally refine a subset of facial landmarks generated by previous network levels. In addition, each level predicts explicit geometric constraints (the position and rotation angles of a specific facial component) to rectify the inputs of the current network level. The combination of coarse-to-fine cascade and geometric refinement enables our system to locate extensive facial landmarks (68 points) accurately in the 300-W facial landmark localization challenge.

Sign-Correlation Partition Based on Global Supervised Descent Method for Face Alignment

“…Ren et al [33] build local binary features by learning regression random forest for each landmark independently, and then learn a global cascaded linear regressor with pre-built binary features. Deep Neural Networks [16,24,25] have also been studied for face landmark detection. DNNs-based methods fuse the feature description and networking training in a unified framework, but it is still a very hard task to tune many free parameters.…”

“…The choice and learning of shape-indexed features are also studied [15][16][17]. A series of regression methods have been employed into cascaded regression framework to deal with over-fitting and local minima problems in the wild condition, including ridge regression [18], Support Vector [19], Gaussian process [20,21], Random Forest voting [14,22,23], Deep Neural Nets [16,24,25], and project-out cascaded regression [26].…”

Robust Face Alignment with Random Forest: Analysis of Initialization, Landmarks Regression, and Shape Regularization Methods

“…Xiong et al [14] proposed Supervised Descent Method to optimize facial feature points search by using SIFT features. Sun et al [15] and Zhou et al [16] also proposed deep convolutional networks for face alignment. Recently, instead of learning a mapping function from image features to landmarks' displacement errors, Tzimiropoulos [17] showed that one can achieve very accurate alignment results by first learning a sequence of averaged Jacobian and Hessian matrices in a subspace orthogonal to the facial appearance variation.…”

Enhancement of Fast Face Detection Algorithm Based on a Cascade of Decision Trees

“…The face detection task is global because it is used in commercial and law enforcement applications (Suri, 2011) and (Cerna, 2013). The task of face detection on real images was created in real conditions, the so-called "faces in-the-wild", is relevant at the moment, despite significant progress in the development of such algorithms (Zhou, 2013), (Li, 2013), (Dalal, 2005), (Riopka, 2003), (Yan, 2014), (Felzenszwalb, 2008) and .…”