Thermal to visible face recognition

Choi, Jonghyun; Hu, Shuowen; Young, S. Susan; Davis, Larry S.

doi:10.1117/12.920330

Cited by 42 publications

(37 citation statements)

References 29 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Statistical significance aside, the results are comparable and in most cases exceed the reported performance of current cross-band facial recognition algorithms. [1][2][3][4][5][6][7] This result confirms our working hypothesis that humans are currently superior to algorithms in performing the cross-band facial authentication task.…”

Section: Discussionsupporting

confidence: 89%

“…This is due in large part to the uncharacteristically low recognition rates being reported from algorithm performance studies. [1][2][3][4][5][6][7] The fact that facial authentication authorities rely on both computer algorithm and human analyst input to make final authentication decisions makes this low performance issue problematic as we, as a community look for ways to increase the utility of current systems, i.e., facilitate cross-band authentication of persons of interest. Understanding the humans ability to perform this task will significantly help improve both algorithm and biometric system performance.…”

Section: Introductionmentioning

confidence: 99%

“…[1][2][3][4][5][6][7] Inherent factors such as camera effects (blur, noise, and sampling), and variation in pose and illumination, are known to negatively affect algorithm performance. Recent success, however, has been demonstrated for the thermal to visible facial recognition problem, i.e., the use of algorithms to match probe images taken in the MWIR spectrum to gallery images taken in the visible spectrum.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Human task performance baseline: results from a cross-band facial identification perception study

Byrd

Choi

2015

SPIE Proceedings

View full text Add to dashboard Cite

Cross-band facial recognition is a difficult task, even for the most robust matching algorithms. Inherent factors such as camera effects (blur, noise, and sampling), and variation in pose and illumination, are known to negatively affect algorithm performance. Because cross-band matching algorithms are in the infancy of development, it is currently unclear if their performance is superior to human observers performing this task. In this paper, we present findings from a pilot study aimed at analyzing the ability of an ensemble of human observers to perform the 1:N cross-band facial identification task on degraded facial images, where the probe and gallery images were captured in different spectral bands (visible, SWIR, MWIR and LWIR). Results from our 11-alternative forced choice perception study indicate that: 1) a group of observers familiar with even a subset of subjects in a gallery set are, on average, able to perform the task with higher probability (p > 0.15) than a group of observers with no prior exposure, and 2) task performance for both the familiar and unfamiliar groups increased 1.5-3.4% when matching multi-spectral probe images to galleries of 24-bit color facial images vs. 8-bit monochrome facial images. For the SWIR case, however, we observed a 9.1% increase in performance with 24-bit facial images vs. 8-bit facial images. Results from this study can be leveraged for future work directly comparing cross-band matching performance of humans vs. algorithms.

show abstract

Section: Discussionsupporting

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Human task performance baseline: results from a cross-band facial identification perception study

Byrd

Choi

2015

SPIE Proceedings

View full text Add to dashboard Cite

show abstract

“…To the best of our knowledge, there are only a few studies addressing thermal-to-visible face recognition, other than our preliminary conference article on the topic [10]. Bourlai et al proposed a MWIR-to-visible face recognition system consisting of three main stages: preprocessing, feature extraction, and matching using similarity measures [11].…”

Section: Background and Related Workmentioning

confidence: 98%

Thermal-to-visible face recognition using partial least squares

Hu¹,

Choi²,

Chan³

et al. 2015

J. Opt. Soc. Am. A

Self Cite

View full text Add to dashboard Cite

Although visible face recognition has been an active area of research for several decades, cross-modal face recognition has only been explored by the biometrics community relatively recently. Thermal-to-visible face recognition is one of the most difficult cross-modal face recognition challenges, because of the difference in phenomenology between the thermal and visible imaging modalities. We address the cross-modal recognition problem using a partial least squares (PLS) regression-based approach consisting of preprocessing, feature extraction, and PLS model building. The preprocessing and feature extraction stages are designed to reduce the modality gap between the thermal and visible facial signatures, and facilitate the subsequent one-vs-all PLS-based model building. We incorporate multi-modal information into the PLS model building stage to enhance cross-modal recognition. The performance of the proposed recognition algorithm is evaluated on three challenging datasets containing visible and thermal imagery acquired under different experimental scenarios: time-lapse, physical tasks, mental tasks, and subject-to-camera range. These scenarios represent difficult challenges relevant to real-world applications. We demonstrate that the proposed method performs robustly for the examined scenarios.

show abstract

“…A number of previous works have proposed various methods to detect faces under varying illumination conditions 3 , but all these methods perform poorly in low light conditions. Conventional thermal and polarimetric thermal imaging are emission dominated, therefore, they are ideal for low-light and night time based face detection techniques 4 . This work focuses on evaluating the feasibility of current face detection techniques on conventional thermal and polarimetric S thermal face imagery, as well as face imagery in the visible spectrum.…”

Section: Introductionmentioning

confidence: 99%

Toward automated face detection in thermal and polarimetric thermal imagery

Gordon

Acosta

Short

et al. 2016

Signal Processing, Sensor/Information Fusion, and Target Recognition XXV

Self Cite

View full text Add to dashboard Cite

Visible spectrum face detection algorithms perform pretty reliably under controlled lighting conditions. However, variations in illumination and application of cosmetics can distort the features used by common face detectors, thereby degrade their detection performance. Thermal and polarimetric thermal facial imaging are relatively invariant to illumination and robust to the application of makeup, due to their measurement of emitted radiation instead of reflected light signals. The objective of this work is to evaluate a government off-the-shelf wavelet based naïve-Bayes face detection algorithm and a commercial off-the-shelf Viola-Jones cascade face detection algorithm on face imagery acquired in different spectral bands. New classifiers were trained using the Viola-Jones cascade object detection framework with preprocessed facial imagery. Preprocessing using Difference of Gaussians (DoG) filtering reduces the modality gap between facial signatures across the different spectral bands, thus enabling more correlated histogram of oriented gradients (HOG) features to be extracted from the preprocessed thermal and visible face images. Since the availability of training data is much more limited in the thermal spectrum than in the visible spectrum, it is not feasible to train a robust multi-modal face detector using thermal imagery alone. A large training dataset was constituted with DoG filtered visible and thermal imagery, which was subsequently used to generate a custom trained Viola-Jones detector. A 40% increase in face detection rate was achieved on a testing dataset, as compared to the performance of a pre-trained/baseline face detector. Insights gained in this research are valuable in the development of more robust multimodal face detectors.

show abstract

Thermal to visible face recognition

Cited by 42 publications

References 29 publications

Human task performance baseline: results from a cross-band facial identification perception study

Human task performance baseline: results from a cross-band facial identification perception study

Thermal-to-visible face recognition using partial least squares

Toward automated face detection in thermal and polarimetric thermal imagery

Contact Info

Product

Resources

About