Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

Abstract

Interest in thermal to visible face recognition has grown significantly over the last decade due to advancements in thermal infrared cameras and analytics beyond the visible spectrum. Despite large discrepancies between thermal and visible spectra, existing approaches bridge domain gaps by either synthesizing visible faces from thermal faces or by learning the cross-spectrum image representations. These approaches typically work well with frontal facial imagery collected at varying ranges and expressions, but exhibit significantly reduced performance when matching thermal faces with varying poses to frontal visible faces. We propose a novel Domain and Pose Invariant Framework that simultaneously learns domain and pose invariant representations. Our proposed framework is composed of modified networks for extracting the most correlated intermediate representations from off-pose thermal and frontal visible face imagery, a sub-network to jointly bridge domain and pose gaps, and a joint-loss function comprised of crossspectrum and pose-correction losses. We demonstrate efficacy and advantages of the proposed method by evaluating on three thermalvisible datasets: ARL Visible-to-Thermal Face, ARL Multimodal Face, and Tufts Face. Although DPIF focuses on learning to match off-pose thermal to frontal visible faces, we also show that DPIF enhances performance when matching frontal thermal face images to frontal visible face images.

A Survey on Thermal to Visual Cross Domain Face Recognition

“…This process amplified the heat signature in the resulting thermal image. In [111], they introduce a framework that learns domain and pose invariant representations. It includes networks for extracting correlated intermediate features from thermal and frontal visible face images and a subnetwork to bridge domain and pose differences.…”

Mapping Stakeholder Perceptions: Navigating Biometric Data Protection Initiative and Face Recognition Technology Support in Indonesia

“…Ethical and privacy concerns are equally important since the prevalence of face data collecting raises concerns about inappropriate surveillance, consent, and illegal data use [42]. In addition, the efficacy of the technology can be impaired by poor lighting, occlusions, or fluctuations in facial expressions, making it less reliable in real-world, uncontrolled conditions [43]- [45]. Face recognition's broad capabilities create concerns regarding the possibility of mass monitoring infringing on civil liberties, making it difficult to balance security imperatives and individual rights [46].…”