Fast, reliable head tracking under varying illumination: an approach based on registration of texture-mapped 3D models

Abstract

ÐAn improved technique for 3D head tracking under varying illumination conditions is proposed. The head is modeled as a texture mapped cylinder. Tracking is formulated as an image registration problem in the cylinder's texture map image. The resulting dynamic texture map provides a stabilized view of the face that can be used as input to many existing 2D techniques for face recognition, facial expressions analysis, lip reading, and eye tracking. To solve the registration problem in the presence of lighting variation and head motion, the residual error of registration is modeled as a linear combination of texture warping templates and orthogonal illumination templates. Fast and stable on-line tracking is achieved via regularized, weighted least-squares minimization of the registration error. The regularization term tends to limit potential ambiguities that arise in the warping and illumination templates. It enables stable tracking over extended sequences. Tracking does not require a precise initial fit of the model; the system is initialized automatically using a simple 2D face detector. The only assumption is that the target is facing the camera in the first frame of the sequence. The formulation is tailored to take advantage of texture mapping hardware available in many workstations, PCs, and game consoles. The nonoptimized implementation runs at about 15 frames per second on a SGI O2 graphic workstation. Extensive experiments evaluating the effectiveness of the formulation are reported. The sensitivity of the technique to illumination, regularization parameters, errors in the initial positioning, and internal camera parameters are analyzed. Examples and applications of tracking are reported. Index TermsÐVisual tracking, real-time vision, illumination, motion estimation, computer human interfaces.

Automatic Facial Expression Recognition

“…Depending on the approach the morphing may use planar models of the face [34,29], cylindrical models of the face [27], ellipsoid models [38], 2D active appearance models based on a triangulated mesh [32], or 3D deformable models [34,5,55].…”

Active blobs: region-based, deformable appearance models

“…A nonlinear minimization procedure is iterated until the closest possible match is found. This basic registration approach has been used in various applications such as mosaicing [51], tracking [7,65], image enhancement/super-resolution [16,23], and computer-human interfaces [10,24]. To date, most registration-based methods require off-line processing, although multi-scale techniques offer some hope for realtime performance [23,52].…”

Low-Rank Representation with Graph Constraints for Robust Visual Tracking

“…In recent years, model based tracking algorithms have received significant attention due to their good performance of handling appearance variability of the target object, such as 3D models [1], integration of shape and color [2], foreground/background models [3], kernel-based filters [4] and subspace learning models [5], [6]. The common part of these algorithms is that all of them build or learn a model of the target object at first and then use it for tracking.…”