A lightweight convolutional swin transformer with cutmix augmentation and CBAM attention for compound emotion recognition

Facial emotion recognition has become a complicated task due to individual variations in facial characteristics, as well as racial and cultural variances. Different psychological studies show that there are complex expressions other than basic emotions which are made up of two basic emotions like"Happily Disgusted", "Happily Surprised", "Sadly Surprised", etc. Compound emotion recognition is challenging due to very less publicly available compound emotion datasets which are imbalanced too. In this paper, we have proposed an LSwin-CBAM for the classification of compound emotions. To address the problem of the imbalanced dataset, the proposed model exploits the cutmix augmentation technique for data augmentation. It also incorporates the CBAM attention mechanism to emphasize the relevant features in an image and swin transformer with fewer swin transformer blocks which leads to less computational complexity in terms of trainable parameters and improves the overall classification accuracy as well. The experimental results of LSwin-CBAM on RAF-DB and EmotioNet datasets show that the proposed transformer-based network can well recognize compound emotions.