Explainable Thermal to Visible Face Recognition Using Latent-Guided Generative Adversarial Network

Abstract

One of the main challenges in performing thermalto-visible face image translation is preserving the identity across different spectral bands. Existing work does not effectively disentangle the identity from other confounding factors. In this paper, we propose a Latent-Guided Generative Adversarial Network (LG-GAN) to explicitly decompose an input image into identity code that is spectral-invariant and style code that is spectral-dependent. By using such a disentanglement, we are able to analyze the identity preservation by interpreting and visualizing the identity code. We present extensive face recognition experiments on two challenging Visible-Thermal face datasets. We show that the learned identity code is effective in preserving the identity, thus offering useful insights on interpreting and explaining thermal-to-visible face image translation.

Thermal–Visible Face Recognition Based on CNN Features and Triple Triplet Configuration for On-the-Move Identity Verification

“…Anghelone et al [14] proposed a Latent-Guided Generative Adversarial Network (LG-GAN) to decompose images into an identity latent code and a style latent code. It allows spectral-invariant and spectral-dependent properties to be obtained.…”

Attention-Guided Generative Adversarial Network for Explainable Thermal to Visible Face Recognition

“…Prior works have involved notably Pix2Pix [13], aimed at learning to map a conditional input thermal image to an output visible image. The optimization step was further regularized by introducing additional constraints such as closed-set face recognition losses [21,17] or face verification losses [4,2], in order to preserve the identity mapping. In comparison to these, some other recent works have focused on preserving the attribute mapping by using a pre-trained attribute prediction network [12,8].…”

“…Explainable GANs aim to empower image translation by transparency and interpretability. Recent works predominantly focused on the visualization and understanding of internal representations [14,2]. Kim et al [14] incorporated learnable attention modules into the generator and the discriminator for unsupervised image-to-image translation.…”

“…Image-to-image translation has received increased attention due to great progress in the field of generative adversarial networks (GANs) [22,13,14,16,15]. We here are interested in thermal-to-visible image synthesis via conditional adversarial networks, which represents the task of generating photo-realistic visible face images conditioned on certain input thermal data [4,8,2]. This task has a wide range of applications including cross-spectral face recognition [2,4] and face landmark detection [19], highly pertinent in defense, surveillance and public safety.…”

“…However, all named works did not offer insightful explanation or visualization on which type of axial-attention or selfattention features were learned during the thermal to visible generation process. Recently, Anghelone et al [2] utilized two separate identity and style encoders to disentangle the latent space into identity and style code representations. The associated visualization of identity code demonstrated that the identity-related structure information were well preserved during the translation.…”