Robust Face Recognition From Multi-View Videos

Abstract

Abstract-Multi-view face recognition has become an active research area in the last few years. In this paper, we present an approach for video-based face recognition in camera networks. Our goal is to handle pose variations by exploiting the redundancy in multi-view video data. However, unlike traditional approaches that explicitly estimate the pose of the face, we propose a novel feature for robust face recognition in the presence of diffuse lighting and pose variations. The proposed feature is developed using the spherical harmonic representation of the face texture-mapped onto a sphere; the texture map itself is generated by back-projecting the multi-view video data. Video plays an important role in this scenario. First, it provides an automatic and efficient way for feature extraction. Second, the data redundancy renders the recognition algorithm more robust. We measure the similarity between feature sets from different videos using the Reproducing Kernel Hilbert Space. We demonstrate that the proposed approach outperforms traditional algorithms on a multi-view video database.

Graph-Based Multi-Modal Multi-View Fusion for Facial Action Unit Recognition

“…The use of multi-view data has become a promising approach to handle the inherent challenges brought by pose variations [20]. The term multi-view data refers to data collected by multiple cameras at different viewpoints simultaneously.…”

“…By utilizing multiple viewpoints, the disadvantages of a single viewpoint are mitigated since the system has access to more information. In the study of face recognition, it has been demonstrated that the fusion of multiview face images can improve recognition accuracy [20] [35]. Researchers have also employed graph-based approaches in multi-modal fusion, using graphs to model the relationships between different modalities and their features.…”

“…It has been shown that fusing different image modalities such as thermal imaging, near-infrared imaging and depth maps can provide more robust and accurate facial expression recognition results concerning variations in illumination than using RGB images alone [18] [19]. Furthermore, it has been shown in the area of face recognition that utilizing multi-view data captured by multiple cameras simultaneously is an effective method for addressing pose variations and their inherent challenges [20]. However, no studies to date have investigated the effectiveness of utilizing multi-modal or multi-view models for AU detection, particularly in the context of real-world applications such as ADAS.…”

Multi-view face recognition from single RGBD models of the faces

“…At test time, faces are recognized by maximizing the posterior probabilities derived from the camera and the human subject nodes. Du et al (2014) aggregate evidence in multi-camera scenarios by tracking a human head from camera to camera. The head model used in this work is a texture-mapped sphere that is represented by spherical harmonics.…”

“…The general problem of recognizing faces under unconstrained conditions remains largely unsolved even for seemingly easy scenarios such as when there is sufficient illumination, the motion of the human subject is slow compared to the camera frame rate, and when high resolution cameras are employed. A solution to this general problem would be relevant in a number of applications, which include face verification and identification in static imagery (Abate, Nappi, Riccio, Sabatino, 2007, Phillips, Grother, Micheals, 2011, Zhao, Chellappa, Phillips, Rosenfeld, 2003, video (Krueger, Zhou, 2002, Lee, Ho, Yang, Kriegman, 2003, and with camera networks (An, Bhanu, Yang, 2012, Du, Sankaranarayanan, Chellappa, 2014.…”