Single view-based 3D face reconstruction robust to self-occlusion

Abstract

State-of-the-art 3D morphable model (3DMM) is used widely for 3D face reconstruction based on a single image. However, this method has a high computational cost, and hence, a simplified 3D morphable model (S3DMM) was proposed as an alternative. Unlike the original 3DMM, S3DMM uses only a sparse 3D facial shape, and therefore, it incurs a lower computational cost. However, this method is vulnerable to self-occlusion due to head rotation. Therefore, we propose a solution to the self-occlusion problem in S3DMM-based 3D face reconstruction. This research is novel compared with previous works, in the following three respects. First, self-occlusion of the input face is detected automatically by estimating the head pose using a cylindrical head model. Second, a 3D model fitting scheme is designed based on selected visible facial feature points, which facilitates 3D face reconstruction without any effect from self-occlusion. Third, the reconstruction performance is enhanced by using the estimated pose as the initial pose parameter during the 3D model fitting process. The experimental results showed that the self-occlusion detection had high accuracy and our proposed method delivered a noticeable improvement in the 3D face reconstruction performance compared with previous methods.

A Comprehensive Survey on Pose-Invariant Face Recognition

“…This is because the position of facial contour in the 2D image changes along with pose variations. To handle this problem, (Lee et al 2012;Qu et al 2014) proposed to detect and discard the moved landmarks. Asthana et al (2011b) proposed the construction of a lookup table which contains the manually labeled 3D vertices that correspond to the 2D facial landmarks under a set of discrete poses.…”

Monocular face reconstruction with global and local shape constraints

“…We will compare the accuracy of the proposed feature recovery algorithm based on global shape constraint with that of existing approach [13], and show the strong adaptability of the proposed algorithm on variant poses of input face images taken from persons with different ages, races and genders. The following errors…”

“…The average feature recovery error ϵ S is 0.0388, among the 100 tests. We compare the proposed algorithm with alternating least square algorithm for 3D feature recovery [13]. The errors of both algorithms are shown in Table 3, in which A ϵ S i and P ϵ S i represent alternating least square algorithm and the proposed algorithm respectively.…”

“…In addition, these improvements fail to consider some powerful methods such as sparse representation [15,16] and novel distance metric learning [17][18][19][20], which have been proved effective in solving linear approximation problems. Most importantly, the shape constraint here is also global shape constraint, and some approaches [13] can only recover a few 3D feature points, 3D face model should be generated by warping a high-resolution reference 3D face using the feature points. Prevalent warping method is scattered data interpolation [21] which relies on measuring Euclidean distances directly between feature points and landmark model points distributed on the model surface, therefore the warping is still conducted in a global way and rarely achieves ideal result with few feature points.…”

“…Instead of using all 3D points, some researchers compute the model parameters by fitting a few salient 3D facial feature points to corresponding image feature points [11][12][13][14]. The feature-based fitting speeds up the computation process enormously, but the parameter estimation of the above-mentioned works is unsatisfactory because it is based on alternating least square method which is not derived from the conditions driving the objective Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/neucom function to reach its optimal value.…”