A Multi-modal Approach for Emotion Recognition Through the Quadrants of Valence–Arousal Plane

Emotion recognition has become a popular area of research in recent years, as emotions play a significant role in our social lives. There are many internal and external factors that trigger emotional changes, making it important to develop effective methods for identifying and managing emotional states. One such method is the use of physiological signals, which has revolutionized the treatment and diagnosis of mental health conditions. In this work, the authors focus on using electroencephalogram (EEG) and electrocardiogram (ECG) signals collected from a standard dataset to classify four emotional states based on the Arousal–Valence plane. The states considered are High Arousal and Low Valence (HALV), Low Arousal and Low Valence (LALV), High Arousal and High Valence (HAHV), and Low Arousal and High Valence (LAHV). The signals were collected non-invasively using sensors, and different standard machine intelligence algorithms were used to classify the emotional states. The experiments were conducted in two phases, and the results showed that the k-Nearest Neighbors algorithm was effective in handling class-imbalanced data, while the Logistic Regression algorithm outperformed other algorithms with an F1 score of 53.5% when trained with class-balanced data. This study presents a novel approach to emotion recognition and provides important insights into the use of physiological signals for identifying emotional states. © 2023, The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd.