Bidirectional Relighting for 3D-Aided 2D Face Recognition

Abstract

In this paper, we present a new method for bidirectional relighting for 3D-aided 2D face recognition under large pose and illumination changes. During subject enrollment, we build subject-specific 3D annotated models by using the subjects' raw 3D data and 2D texture. During authentication, the probe 2D images are projected onto a normalized image space using the subject-specific 3D model in the gallery. Then, a bidirectional relighting algorithm and two similarity metrics (a view-dependent complex wavelet structural similarity and a global similarity) are employed to compare the gallery and probe. We tested our algorithms on the UHDB11 and UHDB12 databases that contain 3D data with probe images under large lighting and pose variations. The experimental results show the robustness of our approach in recognizing faces in difficult situations.

The impact of specular highlights on 3D-2D face recognition

“…In comparison, 3D-2D face recognition systems require a subject to enroll once at a site with full 3D equipment, but authentication is very flexible, as 2D imaging systems are widely spread and straightforward to set up. In particular, we use the 3D-2D face recognition system by Toderici et al 8 . This system has two stages:…”

“…Similarly to the enrollment phase, a visibility map is computed. Differences in the illumination conditions of the probe are normalized by an optimization-based Figure 1: The face verification phase: From the fitted and pose aligned mesh, the texture is lifted and compared with the relit gallery texture before a score is computed between the textures (image courtesy of Toderici et al 8 ).…”

“…The unknown albedo c d depends only on the diffuse reflectance component. Within the framework of Toderici et al 8 , it is implicitly computed within the relighting step. Three lights were used as proposed by Toderici et al.…”

“…To demonstrate the impact of specular highlights, we use the 3D-2D FR system proposed by Toderici et al 8 .…”

Minimum Variance Method to Obtain Best shot in Video for Face Recognition

“…Another method using 3D mesh and the distance function in a wavelet-transformed domain under bad lighting conditions presented by Toderici [9] outperforms the 2D case. However, the recognition ratio is not so high.…”