<title>Infrared and visible image fusion for face recognition</title>

Abstract

Considerable progress has been made in face recognition research over the last decade especially with the development of powerful models of face appearance (i.e., eigenfaces). Despite the variety of approaches and tools studied, however, face recognition is not accurate or robust enough to be deployed in uncontrolled environments. Recently, a number of studies have shown that infrared (IR) imagery offers a promising alternative to visible imagery due to its relative insensitive to illumination changes. However, IR has other limitations including that it is opaque to glass. As a result, IR imagery is very sensitive to facial occlusion caused by eyeglasses. In this paper, we propose fusing IR with visible images, exploiting the relatively lower sensitivity of visible imagery to occlusions caused by eyeglasses. Two different fusion schemes have been investigated in this study: (1) imagebased fusion performed in the wavelet domain and, (2) feature-based fusion performed in the eigenspace domain. In both cases, we employ Genetic Algorithms (GAs) to find an optimum strategy to perform the fusion. To evaluate and compare the proposed fusion schemes, we have performed extensive recognition experiments using the Equinox face dataset and the popular method of eigenfaces. Our results show substantial improvements in recognition performance overall, suggesting that the idea of fusing IR with visible images for face recognition deserves further consideration.

Fusion of Visible and Infrared Images using Empirical Mode Decomposition to Improve Face Recognition

“…Image fusion of multiple imaging modalities has been performed in various ways in the literature. Singh et al [1] use PCA to decompose images into components for fusion using a genetic algorithm. Fusion is performed by selecting the maximum of the visible and infrared image wavelet coefficients by Li et al [2].…”

Feature Level Fusion of Face and Fingerprint Biometrics

“…This technique aims at finding the percentage of points "paired" between the concatenated feature pointset of the database and the query images. Two points are considered paired only if the spatial distance (1), the direction distance (2) and the Euclidean distance (3) between the corresponding key descriptors are all within some are within a pre-determined threshold, set with 4 pixels, 30, 6 pixels for r0, O0, ko on the basis of experiments: sd (concat>,concat) =(X Xi)2 + (Y -t)2 <r0 (1) dd(concat', concat,) = min(6! -6t ,3600 -|6 -6t) < 00 kd (concat j,concat i) = (k k)2 < ko (2) concat' and concat, sd is the spatial distance, dd is the direction distance, and kd is the keypoint descriptor distance.…”

“…This results in a fused feature pointset concat=(Slnorm , S2norm, Smnorm mlnorm, m2norm, mmnorm). Feature reduction strategy to eliminate irrelevant features can be applied either before [7] or after [5][6] Redundant features are then removed using the "k-means" clustering techniques [12] on the fused pointset of an individual retaining only the centroid of the points from each cluster. These clusters are formed using spatial and orientation information of a point.…”

“…Gyaourova et al [6] The face recognition system, preliminary introduced in [8], is based on the SIFT [9] features extracted from images of the query and database face. The SIFT features represent a compact representation of the local gray level structure, invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projections.…”

An Evaluation of Bimodal Recognition Systems Based on Voice and Facial Images

“…Other useful examples of sensor level fusion are discussed in [26][27][28], where visible and thermal infrared face images at sensor level are fused. By using IR images in conjunction with visible images, illumination challenges in facial Chapter 2.…”