3D Face Alignment via Cascade 2D Shape Alignment and Constrained Structure from Motion

In this paper, we consider fitting a 3D wireframe face model to continuous video sequences for the tasks of simultaneous tracking of rigid head motion and non-rigid facial animation. We propose a two-level integrated model for accurate 3D face alignment. At the low level, the 2D shape is accurately extracted via a regularized shape model relied on a. cascaded parameter/constraint prediction and optimization. At the high level, those already accurately aligned points from the low level are used to constrain the projected 3D wireframe alignment. Using a steepest descent approach, the algorithm is able to extract simultaneously the parameters related to the face expression and to the 3D posture. Extensive fitting and tracking experiments demonstrate the feasibility, accuracy and effectiveness of the developed methods. A performance evaluation also shows that the proposed methods can outperform the fitting based on an active appearance model search and can tackle many disadvantages associated with such approaches.