CIT-EmotionNet: convolution interactive transformer network for EEG emotion recognition

被引:0
作者
Lu, Wei [1 ,2 ,3 ]
Xia, Lingnan [1 ]
Tan, Tien Ping [2 ]
Ma, Hua [1 ,3 ]
机构
[1] Zhengzhou Railway Vocat & Tech Coll, Henan High Speed Railway Operat & Maintenance Engn, Zhengzhou, Henan, Peoples R China
[2] Univ Sains Malaysia, Sch Comp Sci, George Town, Malaysia
[3] Zhengzhou Univ, Ind Technol Res Inst, Zhengzhou, Henan, Peoples R China
关键词
Affective computing; Electroencephalogram (EEG); Emotion recognition; Convolutional neural network (CNN); Transformer;
D O I
10.7717/peerj-cs.2610
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Emotion recognition is a significant research problem in affective computing as it has a lot of potential areas of application. One of the approaches in emotion recognition uses electroencephalogram (EEG) signals to identify the emotion of a person. However, effectively using the global and local features of EEG signals to improve the performance of emotion recognition is still a challenge. In this study, we propose a novel Convolution Interactive Transformer Network for EEG Emotion Recognition, known as CIT-EmotionNet, which efficiently integrates the global and local features of EEG signals. We convert the raw EEG signals into spatial-spectral representations, which serve as the inputs into the model. The model integrates convolutional neural network (CNN) and Transformer within a single framework in a parallel manner. We propose a Convolution Interactive Transformer module, which facilitates the interaction and fusion of local and global features extracted by CNN and Transformer respectively, thereby improving the average accuracy of emotion recognition. The proposed CIT-EmotionNet outperforms state-of-the-art methods, achieving an average recognition accuracy of 98.57% and 92.09% on two publicly available datasets, SEED and SEED-IV, respectively.
引用
收藏
页数:31
相关论文
共 54 条
[1]   Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers [J].
Atkinson, John ;
Campos, Daniel .
EXPERT SYSTEMS WITH APPLICATIONS, 2016, 47 :35-41
[2]   Linking Multi-Layer Dynamical GCN With Style-Based Recalibration CNN for EEG-Based Emotion Recognition [J].
Bao, Guangcheng ;
Yang, Kai ;
Tong, Li ;
Shu, Jun ;
Zhang, Rongkai ;
Wang, Linyuan ;
Yan, Bin ;
Zeng, Ying .
FRONTIERS IN NEUROROBOTICS, 2022, 16
[3]   CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification [J].
Chen, Chun-Fu ;
Fan, Quanfu ;
Panda, Rameswar .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :347-356
[4]   Accurate EEG-Based Emotion Recognition on Combined Features Using Deep Convolutional Neural Networks [J].
Chen, J. X. ;
Zhang, P. W. ;
Mao, Z. J. ;
Huang, Y. F. ;
Jiang, D. M. ;
Zhang, Andy N. .
IEEE ACCESS, 2019, 7 :44317-44328
[5]   Emotion recognition using hierarchical spatial-temporal learning transformer from regional to global brain [J].
Cheng, Cheng ;
Liu, Wenzhe ;
Feng, Lin ;
Jia, Ziyu .
NEURAL NETWORKS, 2024, 179
[6]   Spatio-Temporal Representation of an Electoencephalogram for Emotion Recognition Using a Three-Dimensional Convolutional Neural Network [J].
Cho, Jungchan ;
Hwang, Hyoseok .
SENSORS, 2020, 20 (12) :1-18
[7]   Cross-Subject Multimodal Emotion Recognition Based on Hybrid Fusion [J].
Cimtay, Yucel ;
Ekmekcioglu, Erhan ;
Caglar-Ozhan, Seyma .
IEEE ACCESS, 2020, 8 :168865-168878
[8]   A comparative analysis of machine learning methods for emotion recognition using EEG and peripheral physiological signals [J].
Doma, Vikrant ;
Pirouz, Matin .
JOURNAL OF BIG DATA, 2020, 7 (01)
[9]  
Fei Shen, 2019, 2019 IEEE 4th International Conference on Image, Vision and Computing (ICIVC), P247, DOI 10.1109/ICIVC47709.2019.8981065
[10]   STransFuse: Fusing Swin Transformer and Convolutional Neural Network for Remote Sensing Image Semantic Segmentation [J].
Gao, Liang ;
Liu, Hui ;
Yang, Minhang ;
Chen, Long ;
Wan, Yaling ;
Xiao, Zhengqing ;
Qian, Yurong .
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2021, 14 (14) :10990-11003