TV-GAN: Generative Adversarial Network Based Thermal to Visible Face Recognition

Abstract

This work tackles the face recognition task on images captured using thermal camera sensors which can operate in the non-light environment. While it can greatly increase the scope and benefits of the current security surveillance systems, performing such a task using thermal images is a challenging problem compared to face recognition task in the Visible Light Domain (VLD). This is partly due to the much smaller amount number of thermal imagery data collected compared to the VLD data. Unfortunately, direct application of the existing very strong face recognition models trained using VLD data into the thermal imagery data will not produce a satisfactory performance. This is due to the existence of the domain gap between the thermal and VLD images. To this end, we propose a Thermal-to-Visible Generative Adversarial Network (TV-GAN) that is able to transform thermal face images into their corresponding VLD images whilst maintaining identity information which is sufficient enough for the existing VLD face recognition models to perform recognition. Some examples are presented in Figure 1. Unlike the previous methods, our proposed TV-GAN uses an explicit closed-set face recognition loss to regularize the discriminator network training. This information will then be conveyed into the generator network in the forms of gradient loss. In the experiment, we show that by using this additional explicit regularization for the discriminator network, the TV-GAN is able to preserve more identity information when translating a thermal image of a person which is not seen before by the TV-GAN.

Non-Visual to Visual Translation for Cross-Domain Face Recognition

“…Recently, many approaches [12], [47] based on the generative adversarial networks (GANs) proposed by Goodfellow et al [48] made it possible to obtain a more photo-realistic synthesis image than the existing methods. Additionally, other researchers [49], [50] have employed the GANs to generate VIS face images from TIR face images. Song et al [51] proposed domain-invariant feature learning by generating a VIS face image from a NIR face image using GANs.…”

Heterogeneous Face Recognition Using Domain Specific Units

“…The identification was carried out using the Visual Geometry Group (VGG) network [15] embeddings and achieving an average Equal Error Rate of 34.58%. Similarly, Zhang et al in [16] also proposed a strategy based on GANs to synthesize thermograms to visual light images for further comparison using the VGG embeddings. Experiments using the the Iris dataset [17] (with 29 subjects in total) showed a rank one recognition rate of 19%.…”

Polarimetric Thermal to Visible Face Verification via Attribute Preserved Synthesis

“…Unlike the above mentioned traditional methods, synthesis-based thermal to visible face verification algorithms leverage the synthesized visible faces for verification. Due to the success of CNNs and recently introduced generative adversarial networks (GANs) in synthesizing re-alistic images, various deep learning-based approaches have been proposed in the literature for thermal to visible face synthesis [26,36,39,27]. For example, Riggan et al [27] proposed a two-step procedure (visible feature estimation and visible image reconstruction) to solve the thermalvisible verification problem.…”