Towards 3D Face Recognition in the Real: A Registration-Free Approach Using Fine-Grained Matching of 3D Keypoint Descriptors

Abstract

Registration algorithms performed on point clouds or range images of face scans have been successfully used for automatic 3D face recognition under expression variations, but have rarely been investigated to solve pose changes and occlusions mainly since that the basic landmarks to initialize coarse alignment are not always available. Recently, local feature-based SIFT-like matchCommunicated by C. Schnörr.ing proves competent to handle all such variations without registration. In this paper, towards 3D face recognition for real-life biometric applications, we significantly extend the SIFT-like matching framework to mesh data and propose a novel approach using fine-grained matching of 3D keypoint descriptors. First, two principal curvature-based 3D keypoint detectors are provided, which can repeatedly identify complementary locations on a face scan where local curvatures are high. Then, a robust 3D local coordinate system is built at each keypoint, which allows extraction of pose-invariant features. Three keypoint descriptors, corresponding to three surface differential quantities, are designed, and their featurelevel fusion is employed to comprehensively describe local shapes of detected keypoints. Finally, we propose a multitask sparse representation based fine-grained matching algorithm, which accounts for the average reconstruction error of probe face descriptors sparsely represented by a large dictionary of gallery descriptors in identification. Our approach is evaluated on the Bosphorus database and achieves rankone recognition rates of 96.56, 98.82, 91.14, and 99.21 % on the entire database, and the expression, pose, and occlusion subsets, respectively. To the best of our knowledge, these are the best results reported so far on this database. Additionally, good generalization ability is also exhibited by the experiments on the FRGC v2.0 database.

Boosting 3D LBP-Based Face Recognition by Fusing Shape and Texture Descriptors on the Mesh

“…For the Pitch, and Occlusion subsets our scores are reasonably close to [13], whereas the Cross subset score is a bit distant. The most critical case for our solution is represented by the Yaw subset, where we obtain an accuracy of about 75% for the ⟨S F, H + G L⟩ variant of α 1 .…”

“…First we notice that, despite the fusion scheme, the 3D-LBP on the depth image scores quite below the other methods, for both histogram and score fusion variants. We can notice that our method neatly outperforms [12], [14], while it competes well with [13], equating and outperforming it at several subsets, noticeably at the Disgust and Surprise for expressions, LFAU for action units, and at the Occlusion subset.…”

“…In [12], Lin et al used mesh-SIFT to detect feature points on 3D face scans; Then, the quasi-daisy local shape descriptor [15] at each feature point was obtained using multiple order histograms of differential quantities extracted from the surface; Finally, these local descriptors were matched by computing their orientation angles. The same authors extended this work in [13], by boosting the keypoints matching with the Sparse Representation based Classifier (SRC) [16]. In [14], Berretti et al used a similar paradigm by considering different varieties of histogram descriptors computed at mesh-DOG 3D keypoints [17].…”

“…On this dataset, we can also compare our approach with respect to state of the art methods. In particular, we compared with Li et al [12], Berretti et al [14], and Li et al [13], which share the idea of using keypoints matching, and use the same experimental protocol. Actually, differently from our solution, in these methods keypoints are regarded as points on the meshmanifold, which are stable over multi-scale differentiation, and which are usually detected using the mesh-DOG operator [17].…”

“…The keypoints matching is also improved using the RANSAC algorithm. In their second version, Li et al [13], boosted the keypoints matching with the Sparse Representation based Classifier (SRC) [16]. The approaches of Sandbach [26] and Bayramoglu [28] used also the Bosphorus database, but their purpose and setting are different from ours.…”