Landmark Localisation in 3D Face Data

Abstract

Accurate landmark localisation is an essential precursor to many 3D face processing algorithms but, as yet, there is a lack of convincing solutions that work well over a wide range of head poses.In this thesis, an investigation to localise facial landmarks from 3D images is presented, without using any assumption concerning facial pose. In particular, this research devises new surface descriptors, which are derived from either unstructured face data, or a radial basis function (RBF) model of the facial surface.A ground-truth of eleven facial landmarks is collected over well-registered facial images in the Face Recognition Grand Challenge (FRGC) database. Then, a range of feature descriptors of varying complexity are investigated to illustrate repeatability and accuracy when computed for the full set of eleven facial landmarks. At this stage, the nose-tip and two inner-eye corners are observed as the most distinctive facial landmarks as a tradeoff among repeatability, accuracy, and complexity. Thus, this investigation focuses on the localisation of these three facial landmarks, which is the minimum number of landmarks necessary for pose normalisation.Two new families of descriptors are introduced, namely point-pair and point-triplet descriptors, which require two and three vertices respectively for their computation. Also, two facial landmark localisation methods are investigated; in the first, a binary decision tree is used to implement a cascade filter, in the second, graph matching is implemented via relaxation by elimination. Then, using all of these descriptors and algorithms, a number of systems are designed to localise the nose-tip and two inner-eye corners. Above all, 99.92% of nose-tip landmarks within an accuracy of 12 mm is the best localisation performance, which is achieved by one cascade filter system. Finally, landmark localisation performance is reported by using a novel cumulative error curve. Localisation results are gathered by computing errors of estimated landmark locations against respective ground-truth data.2

Point-Triplet Descriptors for 3D Facial Landmark Localisation

“…The second variant of descriptors are derived from an implicit radial basis function (RBF) model, they are referred to as surface RBF signature (SRS) features, which are related to the previous work in sampling an RBF model [1]. Both variants of descriptors are a natural extension of the previous work in landmark localisation [1]- [5]. The point-triplet descriptors are able to encode surface information within a triangular region defined by a point-triplet into a surface signature, which could be useful not only for 3D face processing but, also, within a number of graph based retrieval applications.…”

“…Then, using contextual support [2], a pair of candidate landmarks is created. As long as SSR value features robustly detect the pronasale landmark, it was found that many candidate pairs of endocanthions can be deleted, as no pronasale landmarks support them [5]. After this, only candidate landmarks with the minimum Mahalanobis distance to the mean of training SSR value features, within a radius of 10 mm, are kept.…”

“…Furthermore, it is clear that this process can be done iteratively. This procedure is referred to as a baricenter sampling points algorithm [5] which motivates the computation of the SRS descriptors: baricenter depth map, 7-bins SRS vector, and SRS histogram, introduced in the following subsections. …”

“…Finally, a 23-bin histogram is produced with the normalised DTS values for each height. In doing this, consistent triangular regions from views at different heights are being sought [5]. The theory being that given a triangular region defined by a point-triplet, an SRS histogram is computed by sampling an RBF surface model at different heights, where such a sampling set is produced using our baricenter sampling point algorithm mentioned in Section 2.2.…”

3D face landmark labelling

“…(We appreciate that this is a slightly easier problem than using the full scan of head and shoulders, and we will address these larger scans in future work.) The landmarks used are a mixture of contributions from [22] and [19].…”