A Synthesis-Based Approach for Thermal-to-Visible Face Verification

Peri, Neehar; Gleason, Joshua; Castillo, Carlos D.; Bourlai, Thirimachos; Patel, Vishal M.; Chellappa, Rama

doi:10.1109/fg52635.2021.9666943

Cited by 14 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most recently, Peri et al [37] made improvements on the ARL-VTF dataset using Contrastive Unpaired Translation (CUT) [34] along with several additional regularizing loss functions: alignment, L1, identity, and arc face [4]. While performance of individual conditions were not reported, the proposed CUT method in [37] achieved 97.7% AUC, 6.9% EER, 77.2% TAR@1%FAR, and 90.5% TAR@5%FAR on average across baseline, expression, and pose conditions. However, on average across all conditions, we achieved 98.99% AUC, 3.38% EER, 88.46% TAR@1%FAR, and 95.41% TAR@5%FAR.…”

Section: A Results On the Arl-vtf Datasetmentioning

confidence: 99%

Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

Fondje

Riggan

2023

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

Interest in thermal to visible face recognition has grown significantly over the last decade due to advancements in thermal infrared cameras and analytics beyond the visible spectrum. Despite large discrepancies between thermal and visible spectra, existing approaches bridge domain gaps by either synthesizing visible faces from thermal faces or by learning the cross-spectrum image representations. These approaches typically work well with frontal facial imagery collected at varying ranges and expressions, but exhibit significantly reduced performance when matching thermal faces with varying poses to frontal visible faces. We propose a novel Domain and Pose Invariant Framework that simultaneously learns domain and pose invariant representations. Our proposed framework is composed of modified networks for extracting the most correlated intermediate representations from off-pose thermal and frontal visible face imagery, a sub-network to jointly bridge domain and pose gaps, and a joint-loss function comprised of crossspectrum and pose-correction losses. We demonstrate efficacy and advantages of the proposed method by evaluating on three thermalvisible datasets: ARL Visible-to-Thermal Face, ARL Multimodal Face, and Tufts Face. Although DPIF focuses on learning to match off-pose thermal to frontal visible faces, we also show that DPIF enhances performance when matching frontal thermal face images to frontal visible face images.

show abstract

Section: A Results On the Arl-vtf Datasetmentioning

confidence: 99%

Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

Fondje

Riggan

2023

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

show abstract

“…The ARL [135] and Tuft [58] data sets are utilized by Di et al [205] to develop a multiscale visible to thermal face verification approach using attribute-guided synthesis. Peri et al [206] collected a data set, named MILAB-VTF(B), for face verification evaluation. The data set consists of matched thermal and visible video recordings.…”

Section: E Face Verificationmentioning

confidence: 99%

Multimodal Face Data Sets—A Survey of Technologies, Applications, and Contents

Al-Mannai,

Al-Thelaya,

Schneider

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Due to their ease-of-use, biometric verification methods to control access to digital devices have become ubiquitous. Many rely on supervised machine learning, a process that is notoriously datahungry. At the same time, biometric data is sensitive from a privacy perspective, and a comprehensive review from a data set perspective is lacking. In this survey, we present a comprehensive review of multimodal face data sets (e.g., data sets containing RGB color plus other channels such as infrared or depth). This follows a trend in both industry and academia to use such additional modalities to improve the robustness and reliability of the resulting biometric verification systems. Furthermore, such data sets open the path to a plethora of additional applications, such as 3D face reconstruction (e.g., to create avatars for VR and AR environments), face detection, registration, alignment, and recognition systems, emotion detection, anti-spoofing, etc. We also provide information regarding the data acquisition setup and data attributes (ethnicities, poses, facial expressions, age, population size, etc.) as well as a thorough discussion of related applications and state-of-the-art benchmarking. Readers may thus use this survey as a tool to navigate the existing data sets both from the application and data set perspective. To existing surveys we contribute, to the best of our knowledge, the first exhaustive review of multimodalities in these data sets.

show abstract

“…Prior works have involved notably Pix2Pix [13], aimed at learning to map a conditional input thermal image to an output visible image. The optimization step was further regularized by introducing additional constraints such as closed-set face recognition losses [21,17] or face verification losses [4,2], in order to preserve the identity mapping. In comparison to these, some other recent works have focused on preserving the attribute mapping by using a pre-trained attribute prediction network [12,8].…”

Section: Related Workmentioning

confidence: 99%

Attention-Guided Generative Adversarial Network for Explainable Thermal to Visible Face Recognition

Chen

Anghelone

Fauré

et al. 2022

2022 IEEE International Joint Conference on Biometrics (IJCB)

View full text Add to dashboard Cite

Thermal to visible face image translation aims at synthesizing high-fidelity visible face images from thermal counterparts, placing emphasis on preserving the identity of the faces. While remarkable progress has been achieved related to the quality of synthetic images, as well as related to associated face matching accuracy, interpreting the generation process from thermal to visible face images remains an open challenge. Towards tackling this challenge, we present a novel generic attention-guided generative adversarial network (AG-GAN) for thermal to visible image translation. The AG-GAN framework is based on an encoder network that directly generates attention feature maps from an input thermal image in either, supervised or unsupervised fashion. A decoder network takes the attention maps and applies adaptive layer-instance normalization, in order to reconstruct the corresponding visible image. We show that solving thermal to visible image translation tasks through AG-GAN significantly improves the cross-spectral face matching accuracy, as well as inherently supports model explanation.

show abstract

A Synthesis-Based Approach for Thermal-to-Visible Face Verification

Cited by 14 publications

References 26 publications

Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

Multimodal Face Data Sets—A Survey of Technologies, Applications, and Contents

Attention-Guided Generative Adversarial Network for Explainable Thermal to Visible Face Recognition

Contact Info

Product

Resources

About