Robust Face Alignment Under Occlusion via Regional Predictive Power Estimation

Abstract

Face alignment has been well studied in recent years, however, when a face alignment model is applied on facial images with heavy partial occlusion, the performance deteriorates significantly. In this paper, instead of training an occlusion-aware model with visibility annotation, we address this issue via a model adaptation scheme that uses the result of a local regression forest (RF) voting method. In the proposed scheme, the consistency of the votes of the local RF in each of several oversegmented regions is used to determine the reliability of predicting the location of the facial landmarks. The latter is what we call regional predictive power (RPP). Subsequently, we adapt a holistic voting method (cascaded pose regression based on random ferns) by putting weights on the votes of each fern according to the RPP of the regions used in the fern tests. The proposed method shows superior performance over existing face alignment models in the most challenging data sets (COFW and 300-W). Moreover, it can also estimate with high accuracy (72.4% overlap ratio) which image areas belong to the face or nonface objects, on the heavily occluded images of the COFW data set, without explicit occlusion modeling.

Multilevel active registration for kinect human body scans: from low quality to high quality

“…The statistical shape model also has been introduced for 3D face reconstruction, face modeling and face animation [4,[41][42][43]. Unlike human body, the facial landmarks [21,[45][46][47] can be detected accurately and used as reliable constraints to initialize the fitting of morphable model. In [25], for aligning two faces, the authors extract the facial features before performing ICP registration.…”

Sign-Correlation Partition Based on Global Supervised Descent Method for Face Alignment

“…L2,1 norm based kernel SVR is presented by Martinez et al [19] to substitute the commonly used least squares regressor, which improves the performance of face alignment across views. Gaussian process [20,21] and Random Forest voting [14,22,23] are also introduced into cascaded regression framework. Zhang et al [34] and Zhu et al [30] further study hierarchical or coarse-to-fine searching for face alignment.…”

“…Some of them can handle partial occlusions [12][13][14]. Some works mainly aim at speeding up the prediction process while keeping high accuracy [13,15].…”

“…The choice and learning of shape-indexed features are also studied [15][16][17]. A series of regression methods have been employed into cascaded regression framework to deal with over-fitting and local minima problems in the wild condition, including ridge regression [18], Support Vector [19], Gaussian process [20,21], Random Forest voting [14,22,23], Deep Neural Nets [16,24,25], and project-out cascaded regression [26].…”

Robust facial landmark detection and tracking across poses and expressions for in-the-wild monocular video

“…Recently, discriminative models have shown promising performance for robust facial landmark detection, represented by cascaded regression-based methods, e.g., explicit shape regression [7], and the supervised descent method [8]. Many recent works following the cascaded regression framework consider how to improve efficiency [9,10] and accuracy, taking into account variations in pose, expression, lighting, and partial occlusion [11,12]. Although previous works have produced remarkable results on nearly frontal facial landmark detection, it is still not easy to locate landmarks across a large range of poses under uncontrolled conditions.…”