Evaluation of multi-modal biometrics using appearance, shape and temperature

This study considers face recognition using multiple imaging modalities. Face recognition is performed using a PCA-based algorithm on each of three individual modalities: normal 2D intensity images, range images representing 3D shape, and infra-red images representing the pattern of heat emission. The algorithm is separately tuned for each modality. For each modality, the gallery consists of one image of each of the same 127 persons. and the probe set consists of 297 images of these subjects, acquired with one or more week's time lapse. In this experiment, we find a rank-one recognition rate of 71% for infra-red, 91% for 2D, 92% for 3D. We also consider the multi-modal combination of each pair of modalities, and find a rank-one recognition rate of 97% for 2D plus infra-red, 98% for 3D plus infra-red, and 99% for 3D plus 2D. The combination of all three modalities yields a rank-one recognition rate of 100%. We conclude that multi-modal face recognition appears to offer great potential for improved accuracy over using a single 2D image. Larger and more challenging experiments are needed in order to explore this potential.