Infrared and visible image fusion for face recognition

Considerable progress has been made in face recognition research over the last decade especially with the development of powerful models of face appearance (i.e., eigenfaces). Despite the variety of approaches and tools studied, however, face recognition is not accurate or robust enough to be deployed in uncontrolled environments. Recently, a number of studies have shown that infrared (IR) imagery offers a promising alternative to visible imagery due to its relative insensitive to illumination changes. However, IR has other limitations including that it is opaque to glass. As a result. IR imagery is very sensitive to facial occlusion caused by eyeglasses. In this paper; we propose fusing IR with visible images, exploiting the relatively lower sensitivity of visible imagery to occlusions caused by eyeglasses. Two different fusion schemes have been investigated in this study: (1) image-based fusion performed in the wavelet domain and, (2) feature-based fusion performed in the eigenspace domain. In both cases. we employ Genetic Algorithms (GAs) to find an optimum strategy to perform the fusion. To evaluate and compare the proposed fusion schemes, we have performed extensive recognition experiments using the Equinox face dataset and the popular method of eigenfaces. Our results show substantial improvements in recognition performance overall, suggesting that the idea of fusing IR with visible images for face recognition deserves further consideration.