Video-based online face recognition using identity surfaces

Abstract

Recognising faces across multiple views is more challenging than that from a $xed view because of the severe non-linearity caused by rotation in depth, self-occlusion, self-shading, and change of illumination. The problem can be related to the problem of modelling the spatiotemporal dynamics of moving faces from video input for unconstrained live face recognition. Both problems remain largely under-developed. To address the problems, a novel approach is presented in this paper: A multi-view dynamic face model is designed to extract the shape-and-pose-free texture patterns of faces. The model provides a precise correspondence to the task of recognition since the 3 0 shape information is used to warp the multi-view faces onto the model mean shape in frontal-view. The identity surface of each subject is constructed in a discriminant feature space from a sparse set of face texture patterns, or more practically, from one or more learning sequences containing the face of the subject. Instead of matching templates or estimating multi-modal density functions, face recognition can be performed by computing the pattern distances to the identity surfaces or trajectory distances between the object and model trajectories. Experimental results depict that this approach provides an accurate recognition rate while using trajectory distances achieves a more robust performance since the trajectories encode the spatio-temporal information and contain accumulated evidence about the moving faces in a video input.

Face Recognition and Retrieval in Video

“…Method Key-frame based Approaches [90], [40], [47], [114], [100], [17], [115], [31], [78], [85], [98], [101], [118] Temporal Model based Approaches [74], [73], [72], [75], [18], [24], [67], [69], [68], [122], [120], [123], [121], [64], [65], [66], [79], [55], [2], [43], [50], [49] Image-Set Matching based Approaches Statistical model-based [93], [4], [96], [7], [10], [6], [9] Mutual subspace-based [110], [90], [35], [82], [108], [56], [57], [5]<...>…”

“…Li et al [74,73,72,75] proposed to model facial dynamics by constructing facial identity structures across views and over time, referred to identity surfaces (shown in Fig. 3), in the Kernel Discriminant Analysis feature space.…”

Far-Field, Multi-Camera, Video-to-Video Face Recognition

“…These algorithms are categorized regarding the way temporal information is used, to report people identities per probe video and not per extracted probe still. There are algorithms based on post-decision fusion (Xie et al, 2004;Stergiou et al, 2006), while others embed the use of temporal information within the face recognizer (Weng et al, 2000;Li et al, 2001;Lee et al, 2003;Liu and Chen, 2003;Raytchev and Murase, 2003;Aggarval et al, 2004). An exception to this categorization can be found in (Gorodnichy, 2003), where temporal information is only utilized in face detection, to provide the best still to attempt recognition.…”

“…Two clustering algorithms are introduced that can lead to unsupervised face recognition. Li et al utilize a pose estimator to fit a multi-view dynamic face model on the video frames (Li et al, 2001). This gives pose invariant textures.…”

“…Given sufficient training data (many gallery stills of each person) and/or high resolution images, the 90% recognition barrier can be exceeded, even for hundreds of different people to be recognized (Phillips et al, 2006). Face recognition on video streams has only recently begun to receive attention (Weng et al, 2000;Li et al, 2001;Gorodnichy, 2003;Lee et al, 2003;Liu and Chen, 2003;Raytchev and Murase, 2003;Aggarval et al, 2004;Xie et al, 2004;Stergiou et al, 2006). Video-to-video face recognition refers to the problem of training and testing face recognition systems using video streams.…”