Facial 3D Shape Estimation from Images for Visual Speech Animation

In this paper we describe the first version of our system for estimating 3D shape sequences from images of the frontal face. This approach is developed with 3D Visual Speech Animation (VSA) as the target application. In particular, the focus is on the usability of an existing state-of-the-art image-based VSA system and subsequent on-line estimation of the corresponding 3D facial shape sequence from its output. This has the added advantage of a 3D visual speech, which is mainly renderability of the face in different poses and illumination conditions. The idea is based on the detection of landmarks from the facial image which are then used to determine the pose and shape. The method belongs to the category of methods which use a prior 3D Morphable Models (3D-MM) trained using 3D facial data. For the time being it is developed for a person-specific domain, i.e. the 3D-MM and the 2D facial landmark detector are trained using the data of a single person and tested with the same person-specific data.