EMCNN: Fine-Grained Emotion Recognition based on PPG using Multi-scale Convolutional Neural Network

The objective nature of physiological electrical signals, which is not susceptible to human manipulation, promotes their application in the field of affective computing. However, most of the existing methods rely on multi-channel or multi-modal signals, which are cumbersome to collect in daily settings, thus limiting their practical application. In this paper, we proposed a photoplethysmography (PPG) based emotion recognition approach that leverages a single-channel signal for fine-grained emotion analysis. To address the inherent simplicity of the PPG signal, an Emotional Multi-scale Convolutional Neural Network (EMCNN) is presented to enrich the metadata by integrating information from both time and frequency domains, thereby enhancing the ability of feature extraction. Moreover, in addition to the binary classification task regarding the aspect of valence or arousal, the 4-dimensional classification task is also considered to achieve fine-grained emotion recognition. Experiments on the DEAP dataset demonstrate that the proposed method obtained outstanding accuracy of 94.2%, 94.0%, and 86.1%, for the binary valence, binary arousal, and 4-dimensional classification, respectively. Furthermore, it exhibits significant generalization ability in achieving considerable performance when tested on the self-collected dataset. The successful implementation of fine-grained PPG-based emotion recognition will not only facilitate the development of non-invasive wearable emotion monitoring but also paves the way for clinical applications.