Robust Facial Feature Tracking

Abstract

We present a robust technique for tracking a set of predetermined points on a human face. To achieve robustness, the Kanade-Lucas-Tomasi point tracker is extended and specialised to work on facial features by embedding knowledge about the configuration and visual characteristics of the face. The resulting tracker is designed to recover from the loss of points caused by tracking drift or temporary occlusion. Performance assessment experiments have been carried out on a set of 30 video sequences of several facial expressions. It is shown that using the original Kanade-Lucas-Tomasi tracker, some of the points are lost, whereas using the new method described in this paper, all lost points are recovered with no or little displacement error.

Facial Feature Tracking and Occlusion Recovery in American Sign Language

“…It is assumed that the nostrils will appear as black circles toward the center of the face, as in Refs. [7] and [22]. Nostril pixels are found by thresholding the intensity values of the face region and computing peaks of the vertical and horizontal histograms of these values.…”

“…Ohtsuki and Healey [19] also base their facial feature extraction system on Farkas' model. Some works [7,22] report robustness in the event of occlusions. Other facial feature trackers [13,21] that also are successful in handling occlusions require training to detect "good" feature points and may not track features useful for sign language recognition.…”

Nostril detection for robust mouth tracking

“…The box size has been heuristically defined according to the image dataset employed. The nostrils are the darkest spots in the middle of the face area [9], so it is possible to locate them using a 5% threshold on the search area histogram [8]. The result of this thresholding is shown in figure 1(c).…”

“…In the literature, several papers on nostril detection are present, e.g. [8], but use an approach proposed by Petajan [9]. They assume the camera is not in at the level of the speaker's face (frontal position), but that it is at a lower level and angled (e.g.…”

Using kernels for a video-based mouse-replacement interface

“…Optical-flow trackers estimate the location of a feature to be tracked by matching the image patch estimated to contain the feature in the previous image with the locally best-matching patch in the current image. Optical-flow trackers are known to incur ''feature drift'' [9]. The tracked location may slowly drift away from the initially-selected feature, for which no record is kept.…”