Cross-spectral face recognition in heterogeneous environments: A case study on matching visible to short-wave infrared imagery

Abstract

In this paper we study the problem of cross spectral face recognition in heterogeneous environments. Specifically we investigate the advantages and limitations of matching short wave infrared (SWIR) face images to visible images under controlled or uncontrolled conditions. The contributions of this work are three-fold. First, three dif f erent databases are considered, which represent three dif f erent data col lection conditions, i.e., images acquired in fully controlled (indoors), semi-controlled (indoors at standoff distances ?: 50m), and uncontrolled (outdoor operational conditions) environments. Second, we demonstrate the possibility of SWIR cross-spectral matching under controlled and chal lenging scenarios. Third, we illustrate how photometric normalization and our proposed cross-photometric score level fusion rule can be utilized to improve cross-spectral matching peiformance across all scenarios. We utilized both commercial and academic (texture-based) face match ers and performed a set of experiments indicating that SWIR images can be matched to visible images with encouraging results. Our experiments also indicate that the level of im provement in recognition performance is scenario depen dent.

Multi-spectral face recognition: Identification of people in difficult environments

“…Parts of this work were reported in [6], [8], [9]. However, in this paper, we provide an overview and a comparative study of several FR challenging scenarios that we have worked on, and we also provide new results when using face images captured at mid-ranges and at night-time environments.…”

“…Automated eye detection was performed when using the visible, short-range NIR (30m), SWIR [9] and MWIR [8] images. When we had to deal with challenging images, traditional face and eye detection techniques did not work, e.g.…”

“…In Figure 2 (c) and (d) we can see two typical cross-spectral matching scenarios where the gallery mug shots were acquired in the visible band under controlled conditions, while the probe images were acquired either in the short-wave [9] or the midwave [8] infrared band respectively. In practice, there are other similar scenarios that can be considered using different bands, e.g.…”

“…First, we use the visible-SWIR datasets and we illustrate how (a) the usage of independent or combined photometric normalization techniques, and (b) cross-photometric score level fusion can be utilized to improve cross-spectral matching performance across all scenarios. This fusion approach tries to bridge the spatial representation of independent gallery (visible) and probe (SWIR) images so that we can achieve higher matching performance [9].…”

Computerized Face Recognition in Renaissance Portrait Art: A quantitative measure for identifying uncertain subjects in ancient portraits

“…Three-dimensional modeling approaches such as [24] have also been studied. Recent research efforts have focused on cross spectral face recognition for comparing images taken in heterogeneous environments [8]. Such methods are not applicable for our study.…”