REAL-TIME 3D FACE RECONSTRUCTION FROM SINGLE IMAGE USING END-TO-END CNN REGRESSION

This paper presents a learning-based method for detailed 3D face reconstruction from a single unconstrained image. The core of our method is an end-to-end multi-task network architecture. The purpose of the proposed network is to predict a geometric representation of 3D face from a given facial image. Unlike most existing reconstruction methods using low-dimension morphable models, we propose a pixel-based multi-scale representation of a detailed 3D face to ensure that our reconstruction results are not limited by the expressiveness of linear models. We break the task of highfidelity face reconstruction into three subtasks, which are face region segmentation, coarse-scale reconstruction and detail recovery. So the end-to-end network is constructed as a multi-task mode, which contains three subtask networks to deal with different subtasks respectively. A backbone network with feature pyramid structure is proposed as well to provide different levels of feature maps required by the three subtask networks. We train our end-to-end network in the spirit of the recent photo-realistic data generation approach. The experimental results demonstrate that our method can work with totally unconstrained images and produce high-quality reconstruction but with less runtime compared to the state-of-the-art.