Coarse-to-fine 3D facial landmark localization based on keypoints

A coarse-to-fine algorithm for 3D facial landmark localization based on keypoints is proposed. The landmark localization is divided into two independent subproblems, that are keypoint detection and labelling. To extract keypoints on 3D faces more effectively, a keypoint detection method is proposed. First, coarse positions of landmarks are located by applying supervised descent method on depth images. The neighborhoods of landmarks' coarse positions are extracted as keypoint regions. Second, a keypoint detection method is achieved by combining multiple local descriptors and filtering out the subset of the facial point set in keypoint regions. At the stage of labelling, a set of landmark candidates are generated from keypoints and those candidates best fitted the facial landmark model are labelled as the landmarks. The proposed algorithm is evaluated on FRGC v2.0 and Bosphorus datasets and compared with several state-of-the-art approaches. On the FRGC v2.0 dataset, the mean errors reach 2.85 mm to 3.81 mm for each landmark. The overall detection success rate is 96.5%, among which 97.5% for neutral expression, 97.0% for mild, 93.3% for extreme. On the Bosphorus dataset, the success rate reaches 92%, 95% and 88% respectively under three different poses. Experimental results show that the presented algorithm achieves good accuracy, efficiency and robustness against expression and small pose variation. © 2018, Science Press. All right reserved.