Bridging the spectral gap using image synthesis: a study on matching visible to passive infrared face images

We propose an approach that bridges the gap between the visible and IR band of the electromagnetic spectrum, namely the mid-wave infrared or MWIR (3-5 mu m) and the long-wave infrared or LWIR (8-14 mu m) bands. Specifically, we investigate the benefits and limitations of using synthesized visible face images from thermal and vice versa, in cross-spectral face recognition systems when utilizing canonical correlation analysis and manifold learning dimensionality reduction. There are four primary contributions of this work. First, we assemble a database of frontal face images composed of paired VIS-MWIR and VIS-LWIR face images (using different methods for pre-processing and registration). Second, we formulate a image synthesis framework and post-synthesis restoration methodology, to improve face recognition accuracy. Third, we explore cohort-specific matching (per gender) instead of blind-based matching (when all images in the gallery are matched against all in the probe set). Finally, by conducting an extensive experimental study, we establish that the proposed scheme increases system performance in terms of rank-1 identification rate. Experimental results suggest that matching visible images against images acquired with passive infrared spectrum, and vice-versa, are feasible with promising results.