Emotion Recognition via Multiscale Feature Fusion Network and Attention Mechanism

被引:11
|
作者
Jiang, Yiye [1 ]
Xie, Songyun [1 ]
Xie, Xinzhou [1 ]
Cui, Yujie [1 ]
Tang, Hao [1 ]
机构
[1] Northwestern Polytech Univ, Sch Elect & Informat, Xian 710072, Shaanxi, Peoples R China
关键词
Feature extraction; Electroencephalography; Emotion recognition; Convolution; Brain modeling; Deep learning; Task analysis; Attention mechanism; deep learning; electroencephalogram (EEG); emotion recognition; feature fusion; spatial-temporal feature; CONVOLUTIONAL NEURAL-NETWORK; FEATURE-EXTRACTION; EEG; CLASSIFICATION;
D O I
10.1109/JSEN.2023.3265688
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Traditional manual feature-based machine learning methods and deep learning networks have been used for electroencephalogram (EEG)-based emotion recognition in recent years. However, some existing studies ignore the low signal-to-noise ratio and the fact that each subject has unique EEG traits, which suffer from low recognition accuracy and poor robustness. To solve these problems, we propose a novel attention mechanism-based multiscale feature fusion network (AM-MSFFN) that considers high-level features at different scales to improve the generalization of the model for different subjects. Specifically, we first utilize a spatial-temporal convolutional block to extract temporal and spatial features of EEG signals sequentially. Subsequently, considering the sampling rate of EEG signals, the multiscale separable convolutions are designed for capturing emotional state-related information, to better combine and output feature mapping relationships. Convolutional module attention mechanism (CBAM) is applied after point-wise convolution, to better handle EEG variations of different subjects and the key information which facilitates classification. In addition, we adopt a preprocessing module based on data augmentation and data alignment to improve the quality of the training samples. Moreover, ablation studies show that the proposed attention mechanism and multiscale separable convolution contribute significant and consistent gain to the performance of our AM-MSFFN model. To verify the effectiveness of the proposed algorithm, we conducted extensive experiments on the DEAP dataset and SEED. The average accuracies achieve 99.479% and 99.297% for arousal and valence, respectively. The results demonstrated the feasibility of the proposed method.
引用
收藏
页码:10790 / 10800
页数:11
相关论文
共 50 条
  • [31] A Multi-Scale Fusion Convolutional Neural Network Based on Attention Mechanism for the Visualization Analysis of EEG Signals Decoding
    Li, Donglin
    Xu, Jiacan
    Wang, Jianhui
    Fang, Xiaoke
    Ji, Ying
    IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 2020, 28 (12) : 2615 - 2626
  • [32] Speech Emotion Recognition via Sparse Learning-Based Fusion Model
    Min, Dong-Jin
    Kim, Deok-Hwan
    IEEE ACCESS, 2024, 12 : 177219 - 177235
  • [33] HASTF: a hybrid attention spatio-temporal feature fusion network for EEG emotion recognition
    Hu, Fangzhou
    Wang, Fei
    Bi, Jinying
    An, Zida
    Chen, Chao
    Qu, Gangguo
    Han, Shuai
    FRONTIERS IN NEUROSCIENCE, 2024, 18
  • [34] Bridge Graph Attention Based Graph Convolution Network With Multi-Scale Transformer for EEG Emotion Recognition
    Yan, Huachao
    Guo, Kailing
    Xing, Xiaofen
    Xu, Xiangmin
    IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2024, 15 (04) : 2042 - 2054
  • [35] A CNN Approach for Emotion Recognition via EEG
    Mahmoud, Aseel
    Amin, Khalid
    Al Rahhal, Mohamad Mahmoud
    Elkilani, Wail S.
    Mekhalfi, Mohamed Lamine
    Ibrahim, Mina
    SYMMETRY-BASEL, 2023, 15 (10):
  • [36] Emotion Recognition Using Three-Dimensional Feature and Convolutional Neural Network from Multichannel EEG Signals
    Chao, Hao
    Dong, Liang
    IEEE SENSORS JOURNAL, 2021, 21 (02) : 2024 - 2034
  • [37] A feature fusion enhanced multiscale CNN with attention mechanism for spot-welding surface appearance recognition
    Xiao, Meng
    Yang, Bo
    Wang, Shilong
    Zhang, Zhengping
    Tang, Xiaoli
    Kang, Ling
    COMPUTERS IN INDUSTRY, 2022, 135
  • [38] Multiscale Feature Interactive Network for Multifocus Image Fusion
    Liu, Yu
    Wang, Lei
    Cheng, Juan
    Chen, Xun
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2021, 70
  • [39] Deep Neural Network for Emotion Recognition Based on Meta-Transfer Learning
    Tang, Hengyao
    Jiang, Guosong
    Wang, Qingdong
    IEEE ACCESS, 2022, 10 : 78114 - 78122
  • [40] A Pre-Activation Residual Convolutional Network With Attention Modules for High-Resolution Segmented EEG Emotion Recognition
    Charalampous, Ioannis
    Mavrokefalidis, Christos
    Berberidis, Kostas
    IEEE ACCESS, 2025, 13 : 16303 - 16313