Automatic Keypoint Detection on 3D Faces Using a Dictionary of Local Shapes

Abstract

Abstract-Keypoints on 3D surfaces are points that can be extracted repeatably over a wide range of 3D imaging conditions. They are used in many 3D shape processing applications; for example, to establish a set of initial correspondences across a pair of surfaces to be matched. Typically, keypoints are extracted using extremal values of a function over the 3D surface, such as the descriptor map for Gaussian curvature. That approach works well for salient points, such as the nosetip, but can not be used with other less pronounced local shapes. In this paper, we present an automatic method to detect keypoints on 3D faces, where these keypoints are locally similar to a set of previously learnt shapes, constituting a 'local shape dictionary'. The local shapes are learnt at a set of 14 manuallyplaced landmark positions on the human face. Local shapes are characterised by a set of 10 shape descriptors computed over a range of scales. For each landmark, the proportion of face meshes that have an associated keypoint detection is used as a performance indicator. Repeatability of the extracted keypoints is measured across the FRGC v2 database.

Comparing 3D Descriptors for Local Search of Craniofacial Landmarks

“…Closer to the present study are the evaluations reported by Romero & Pears [14] and Creusot et al [13]. However, in both cases the evaluation is performed based on the joint search of all points under the global constraints provided by a graph-matching scheme.…”

Compensating inaccurate annotations to train 3D facial landmark localization models

“…For the experiments in this paper we manually annotated 11 landmarks 1 on the first 100 scans from FRGC (v1) and compared their consistency against the publicly available annotations from Szeptycki et al [22], with some additions and corrections introduced by Creusot et al 1 Annotations available at http://fsukno.atspace.eu/Data.htm [4] 2 . This set contains scans from 19 different persons and allows for a total of 248 pairwise comparisons 3 .…”

3D landmark model discovery from a registered set of organic shapes

“…In the experiments presented here, D = 8 local shape descriptors are used, as follows: the first and second principal curvature (k 1 and k 2 ), the Gaussian curvature (K), the mean curvature (H), the Shape Index (SI), the log-curvedness (LC), the local volume (Vol) and the Distance to Local Plane (DLP). These local shape descriptors were computed using implementations provided to us, courtesy of [4].…”

“…Typically, for human faces, extrema of Gaussian curvature have been detected, but this yields a very sparse set of useful keypoints. Others [4] have defined what they believe to be a useful set of locations at which to learn local shape properties, but there is often no obvious geometric justification for these. In many cases it has more to do with the existence of words to describe these locations and the words may even relate to color-texture properties (e.g.…”