Robust Landmark Selection for 3D Face Pose Estimation

Face pose estimation is an important computer vision problem that has many applications. Vehicle driver tracking, augmented reality, human-computer interaction, and face frontalization for recognition are among the examples of applications that benefit face pose estimation. One common way of face pose estimation is solving an optimization problem with given the 2D and 3D landmark locations as input. Estimated face pose should be stable in realtime applications, however jitter occurs due to noise and changes on facial expression. The resulting jitter has a negative effect on pose estimation applications. Therefore, reduction of jitter is an important requirement for face pose estimation. In this study, we aim to detect a set of robust facial landmarks that provides a stable pose estimation. We applied a feature selection scheme by using the variance of rotation vector as the accuracy metric which is computed from frames of face videos. In the experiments, we determined landmark subsets which reduce jitter for test videos with and without facial gestures, and provides a lower variance in rotation vector. As a result of this study, 29 landmark points that are positioned on the face are determined to be the most robust landmarks when the person has no facial expression. When the person has facial expression, 30 landmark points are selected on the face.