3D Facial Landmark Localization Using Combinatorial Search and Shape Regression

Abstract

Abstract. This paper presents a method for the automatic detection of facial landmarks. The algorithm receives a set of 3D candidate points for each landmark (e.g. from a feature detector) and performs combinatorial search constrained by a deformable shape model. A key assumption of our approach is that for some landmarks there might not be an accurate candidate in the input set. This is tackled by detecting partial subsets of landmarks and inferring those that are missing so that the probability of the deformable model is maximized. The ability of the model to work with incomplete information makes it possible to limit the number of candidates that need to be retained, substantially reducing the number of possible combinations to be tested with respect to the alternative of trying to always detect the complete set of landmarks. We demonstrate the accuracy of the proposed method in a set of 144 facial scans acquired by means of a hand-held laser scanner in the context of clinical craniofacial dysmorphology research. Using spin images to describe the geometry and targeting 11 facial landmarks, we obtain an average error below 3 mm, which compares favorably with other state of the art approaches based on geometric descriptors.

Compensating inaccurate annotations to train 3D facial landmark localization models

“…In the SRILF algorithm. The number of combinations to test relates to the number of false positives in the local descriptors and is a measure of the computational complexity associated with the constructed model [20]. Our results show that correction of the training sets also allows reducing the number of tested combinations and therefore the computational complexity.…”

“…To do this, we trained the SRILF algorithm using the corrected annotations {â i } for different values of the uncertainty radius r u . Spin images were used as the local descriptors, following the settings in [20] and using cross-correlation to the corresponding template to compute the similarity scores, as this is the metric originally proposed for spin images [10]. We should emphasize that, although results would change depending on the descriptor, the correction method itself is not restricted to this particular choice.…”

“…The SRILF algorithm [20] combines the response from local feature detectors for each of the targeted landmarks with statistical constraints that ensure the plausibility of landmark positions on a global basis. The algorithm has three components: i) selection of candidates through local feature detection; ii) partial set matching to infer possibly missing landmarks; iii) combinatorial search, which integrates the other two components.…”

“…For this purpose we select the SRILF algorithm (Shape Regression with Incomplete Local Features [20]), which learns both local descriptors and global shape statistics from a training set and has been shown to achieve overall errors of approximately 3 mm on clinical datasets. To test the impact of the corrected annotations, the algorithm is first trained with the original annotations and later trained with corrected annotations for various radii of uncertainty.…”

Feasibility of differential geometry-based features in detection of anatomical feature points on patient surfaces in range image-guided radiation therapy

“…The detection accuracy was compared with those of conventional methods for the detection of facial feature points [17][18][19], and the limitations of this study were discussed.…”