Weighted Patch-based Manifold Regularization Dictionary Pair Learning model for facial expression recognition using Iterative Optimization Classification Strategy

This paper presents a novel method for solving Facial Expression Recognition (FER) task which integrates Manifold Regularization Dictionary Pair Learning and Intra-class Variation Reduced Feature (MRDPL-IVRF) model and Iterative Optimization Classification Strategy (IOCS). In the proposed framework, we firstly divide the facial image into overlapping patches. Weighted Patch-based Local Binary Patterns (WPLBP) is proposed for feature extraction by cascading the weighted LBP features from each patch, which highlights the informative facial regions. MRDPL-IVRF model is then adopted for robust sparse representation of facial features, where IVRFs are generated from the difference between query images and their corresponding approximation of each expression class, which can reduce the intra-class variations resulted by illumination and identity. Finally, IOCS is presented for expression classification which can explore the most unlikely class according to the maximum reconstruction residual. Different from traditional sparse coding classification scheme, our IOCS method iteratively removes irrelevant samples to obtain a more precise representation model, thus can further improve the recognition performance. The feasibility of our framework has been successfully tested on CK, CK+, SEFW, CMU-PIE and Multi-PIE databases. Furthermore, the experiments combining deep features shows that our model can be integrated with existing deep models and obtain better FER results.