NA-Resnet: neighbor block and optimized attention module for global-local feature extraction in facial expression recognition

As deep networks constantly deepen to extract high-level abstract features, the significance of shallow features for the target task will inevitably diminish. To address this issue and provide novel technical support for current research in the field of facial expression recognition (FER), in this article, we propose a network that can increase the decision weight of the shallow and middle feature mappings through the neighbor block (Nei Block) and concentrate on the crucial areas for extracting necessary features through the optimized attention module (OAM), called NA-Resnet. Our work has several merits. First, to the best of our knowledge, NA-Resnet is the first network that directly utilizes surface features to assist image classification. Second, the suggested OAM is embedded into each layer of the network that can precisely extract critical information appropriate to the current stage. Third, our model achieves the best exhibition when using a single relatively lightweight network without a network ensemble on Fer2013. Extensive experiments have been conducted, and the results show that our model achieves much higher state-of-the-art performance than any single network on Fer2013. In particular, our NA-Resnet achieves 74.59% on Fer2013 and an average accuracy of 96.06% with a standard deviation of 2.9% through 10-fold-cross-validation on Ck+.