Real-time facial feature point detection and tracking in a video sequence

Several algorithms for feature detection were compared and the most efficient ones were chosen to detect facial feature points such as eye corners, eyeballs, mouth corners and nostrils. To detect the desired feature points, first face detection was run using Viola-Jones's algorithm that is based on Haar-like features. Having found a face in an image, the face was searched for eyes, nose and mouth. Having found the necessary facial features, those regions of the image containing them were searched for the specific feature points described above. For feature tracking several algorithms were compared as well. As a result of the comparison the algorithm of Lucas-Kanade was chosen to be performed on the detected points. The resultant algorithm detects the feature points to be tracked in the first frame of a video-sequence and then uses these points in the Lucas-Kanade algorithm for tracking. In case more than two points are lost during tracking, feature point detection is run again. This way the algorithm tracks feature points accurately, without lagging in real-time.