The impact of specular highlights on 3D-2D face recognition

Abstract

One of the most popular form of biometrics is face recognition. Face recognition techniques typically assume that a face exhibits Lambertian reflectance. However, a face often exhibits prominent specularities, especially in outdoor environments. These specular highlights can compromise an identity authentication. In this work, we analyze the impact of such highlights on a 3D-2D face recognition system. First, we investigate three different specularity removal methods as preprocessing steps for face recognition. Then, we explicitly model facial specularities within the face detection system with the Cook-Torrance reflectance model. In our experiments, specularity removal increases the recognition rate on an outdoor face database by about 5% at a false alarm rate of 10 −3 . The integration of the Cook-Torrance model further improves these results, increasing the verification rate by 19% at a FAR of 10 −3 .

Deep Highlight Removal Using Temporal Dark Prior in High-Speed Domain

“…Unlike conventional works that propose specular prior on the spatial domain, this study is the first to utilize diffuse priors in the temporal domain. Leveraging the colors of surrounding pixels for hole-filling of highlight removal has already been studied in the non-deep-learning approach [6], [13]. The blur encoder was originally inspired by this conventional method.…”

“…4, it is confirmed that the chromaticity of the saturated region is recovered with the blurred TDP. Reference [6] claims that Gaussian blurred prior can be a helpful cue for the reduction of saturation artifacts and noises in the image. These previous studies show that the surrounding pixels contribute to alleviating hole artifacts.…”

“…Reference [3] removed highlight especially on text image, and improved the performance of text detection and recognition. Also, in face recognition and iris segmentation, highlight removal is an important preprocess [6], [7].…”

Realistic inverse lighting from a single 2D image of a face, taken under unknown and complex lighting

“…where each t i j is an entry of the 3x3 matrix T T T from (3), appeared also in (6). The cost function E( α R , α G , α B ) is regularized with a summation of L2 norms of α i -s in each color channel, primarily, to avoid overfitting.…”

“…Lighting can severely change the appearance of the human face and confound the information which is needed for reconstruction of the face model and face recognition [19] [22] [6]. Not only emitters but also surrounding objects, also from the off-screen space, influence the illumination; objects such as walls, ground, cloths, etc.…”