Geometric Invariants for Facial Feature Tracking with 3D TOF Cameras

Abstract

Abstract-This paper presents a very simple feature-based nose detector in combined range and amplitude data obtained by a 3D time-of-flight camera. The robust localization of image attributes, such as the nose, can be used for accurate object tracking. We use geometric features that are related to the intrinsic dimensionality of surfaces. To find a nose in the image, the features are computed per pixel; pixels whose feature values lie inside a certain bounding box in feature space are classified as nose pixels, and all other pixels are classified as non-nose pixels. The extent of the bounding box is learned on a labeled training set. Despite its simplicity this procedure generalizes well, that is, a bounding box determined for one group of subjects accurately detects noses of other subjects. The performance of the detector is demonstrated by robustly identifying the nose of a person in a wide range of head orientations. An important result is that the combination of both range and amplitude data dramatically improves the accuracy in comparison to the use of a single type of data. This is reflected in the equal error rates (EER) obtained on a database of head poses. Using only the range data, we detect noses with an EER of 0.66. Results on the amplitude data are slightly better with an EER of 0.42. The combination of both types of data yields a substantially improved EER of 0.03.

ToF-sensors: New dimensions for realism and interactivity

“…The recently presented "nose mouse" [9] allows for hands-free text input with a speed of 12 words per minute although it is based on simple geometric features and a limited spatial resolution (176 × 144). An important result is that the robustness of the nose tracker could be drastically increased by using both the intensity and the depth signals of the ToF-camera, compared to using any of the signals alone (see Figure 5).…”

Graffiti Detection Using a Time-Of-Flight Camera

“…Nevertheless, in spite of their limitations, current generation of TOF based depth sensors was successfully used in several applications such as video surveillance and tracking [13], facial features detection and tracking [14], classification of moving objects [15], gesture recognition [16], documentation of heritage objects [11], robot vision and obstacle detection [17,18], safety [12]. Some authors combined TOF sensors with additional high resolution cameras [19,20] in order to overcome the limitations of the embedded CMOS camera or stereo vision systems [21,22] in order to improve effectiveness of depth perception.…”

Facial Feature Tracking Using Efficient Particle Filter and Active Appearance Model

“…In addition, some robust methods used a special sensor rather than a typical camera. A 3D TOF camera was used in [19] and a structured light sensor was used in [20].…”