An appearance model constructed on 3-D surface for robust face recognition against pose and illumination variations

We propose a face recognition method that is robust against image variations due to arbitrary lighting and a large extent of pose variations, ranging from frontal to profile views. Existing appearance models defined on image planes are not applicable for such pose variations that cause occlusions and changes of silhouette. In contrast, our method constructs an appearance model of a three-dimensional (3-D) object on. its surface. Our proposed model consists of a 3-D shape and geodesic illumination bases (GIBs). GIBs can describe the irradiances of an object's surface under any illumination and generate illumination subspace that can describe illumination variations of an image in an arbitrary pose. Our appearance model is automatically aligned to the target image by pose optimization based on a rough pose, and the residual error of this model fitting is used as the recognition score. We tested the recognition performance of our method with an extensive database that includes 14 000 images of 200 individuals with drastic illumination changes and pose variations up to 60 degrees sideward and 45 degrees upward. The method achieved a first-choice success ratio, of 94.2% without knowing precise poses a priori.