Consensus and stacking based fusion and survey of facial feature point detectors

Facial landmark detection is a crucial pre-processing step for many applications including face tracking, face recognition and facial affect recognition. Hence, we first aim to investigate and experimentally compare the most successful open source facial feature point detection algorithms published in the last decade. We first present an overview of surveys on facial feature detection algorithms to provide insight into the challenges and innovations. We also propose a consensus-based selection and stacked regression based fusion of facial landmark methods to combine their results in order to achieve superior accuracy. Five open-source algorithms in the literature are objectively compared using the same test data and regression based models have been shown to be more successful. According to the extensive experimental results, the proposed consensus and stacking based fusion method gives the lowest facial landmark detection error as compared to the five most successful algorithms in the literature. Consensus and stacking based fusion of an ensemble of methods boosts the performance of facial landmark detection. The proposed fusion method can also be applied future methods as they emerge.