Active shape model based segmentation and tracking of facial regions in color images

An approach for segmenting and tracking a face in a sequence of color images is presented. It enables reliable segmentation of facial region despite variation of skin-color perceived by a camera. A second order Markov model is utilized to forecast the skin distribution of facial regions in the next frame. The histograms that are constructed from the predicted distribution are backprojected to generate candidates of facial regions. Afterwards, a connected component labeling takes place. Spatial morphological operations, such as size and hole filtering are employed next. The Active Shape Model seeks to match a set of model points to the image. This statistical model of shape supports the segmentation of facial region undergoing tracking. Histograms are accommodated over time using feedback from shape, newly classified skin pixels and predictions of the skin-color evolution. This evolution is described by translation, rotation and scaling. In this context, the novelty of our approach lies in the introduction of Active Shape Model dealing with translation, rotation and scaling of the target to support face verification as well as to guide the evolution of skin distribution. The kernel histograms characterize the face during tracking in subsequent frames. The proposed algorithm achieves reliable detection and tracking results. The resulting system runs in real-time on standard PC computer.