Multi-source domain adaptation for EEG emotion recognition based on inter-domain sample hybridization

被引:0
作者
Wu, Xu [1 ]
Ju, Xiangyu [1 ]
Dai, Sheng [1 ]
Li, Xinyu [1 ]
Li, Ming [1 ]
机构
[1] Natl Univ Def Technol, Coll Intelligence Sci & Technol, Changsha, Peoples R China
关键词
electroencephalogram; emotion recognition; multi-source domain adaptation; sample hybridization; brain computer interaction;
D O I
10.3389/fnhum.2024.1464431
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Background Electroencephalogram (EEG) is widely used in emotion recognition due to its precision and reliability. However, the nonstationarity of EEG signals causes significant differences between individuals or sessions, making it challenging to construct a robust model. Recently, domain adaptation (DA) methods have shown excellent results in cross-subject EEG emotion recognition by aligning marginal distributions. Nevertheless, these methods do not consider emotion category labels, which can lead to label confusion during alignment. Our study aims to alleviate this problem by promoting conditional distribution alignment during domain adaptation to improve cross-subject and cross-session emotion recognition performance.Method This study introduces a multi-source domain adaptation common-branch network for EEG emotion recognition and proposes a novel sample hybridization method. This method enables the introduction of target domain data information by directionally hybridizing source and target domain samples without increasing the overall sample size, thereby enhancing the effectiveness of conditional distribution alignment in domain adaptation. Cross-subject and cross-session experiments were conducted on two publicly available datasets, SEED and SEED-IV, to validate the proposed model.Result In cross-subject emotion recognition, our method achieved an average accuracy of 90.27% on the SEED dataset, with eight out of 15 subjects attaining a recognition accuracy higher than 90%. For the SEED-IV dataset, the recognition accuracy also reached 73.21%. Additionally, in the cross-session experiment, we sequentially used two out of the three session data as source domains and the remaining session as the target domain for emotion recognition. The proposed model yielded average accuracies of 94.16 and 75.05% on the two datasets, respectively.Conclusion Our proposed method aims to alleviate the difficulties of emotion recognition from the limited generalization ability of EEG features across subjects and sessions. Though adapting the multi-source domain adaptation and the sample hybridization method, the proposed method can effectively transfer the emotion-related knowledge of known subjects and achieve accurate emotion recognition on unlabeled subjects.
引用
收藏
页数:17
相关论文
共 57 条
[1]   Emotions Recognition Using EEG Signals: A Survey [J].
Alarcao, Soraia M. ;
Fonseca, Manuel J. .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2019, 10 (03) :374-393
[2]  
AlShorman O., 2020, 2020 6 INT C E LEARN, P177, DOI [10.1109/econf51404.2020.9385468, DOI 10.1109/ECONF51404.2020.9385468]
[3]  
Ba J, 2014, ACS SYM SER
[4]   Wearable Flexible Electronics Based Cardiac Electrode for Researcher Mental Stress Detection System Using Machine Learning Models on Single Lead Electrocardiogram Signal [J].
Bin Heyat, Md Belal ;
Akhtar, Faijan ;
Abbas, Syed Jafar ;
Al-Sarem, Mohammed ;
Alqarafi, Abdulrahman ;
Stalin, Antony ;
Abbasi, Rashid ;
Muaad, Abdullah Y. ;
Lai, Dakun ;
Wu, Kaishun .
BIOSENSORS-BASEL, 2022, 12 (06)
[5]   Integrating structured biological data by Kernel Maximum Mean Discrepancy [J].
Borgwardt, Karsten M. ;
Gretton, Arthur ;
Rasch, Malte J. ;
Kriegel, Hans-Peter ;
Schoelkopf, Bernhard ;
Smola, Alex J. .
BIOINFORMATICS, 2006, 22 (14) :E49-E57
[6]   Multi-Source and Multi-Representation Adaptation for Cross-Domain Electroencephalography Emotion Recognition [J].
Cao, Jiangsheng ;
He, Xueqin ;
Yang, Chenhui ;
Chen, Sifang ;
Li, Zhangyu ;
Wang, Zhanxiang .
FRONTIERS IN PSYCHOLOGY, 2022, 12
[7]   Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition [J].
Chai, Xin ;
Wang, Qisong ;
Zhao, Yongping ;
Liu, Xin ;
Bai, Ou ;
Li, Yongqiang .
COMPUTERS IN BIOLOGY AND MEDICINE, 2016, 79 :205-214
[8]   MS-MDA: Multisource Marginal Distribution Adaptation for Cross-Subject and Cross-Session EEG Emotion Recognition [J].
Chen, Hao ;
Jin, Ming ;
Li, Zhunan ;
Fan, Cunhang ;
Li, Jinpeng ;
He, Huiguang .
FRONTIERS IN NEUROSCIENCE, 2021, 15
[9]   CONSTANTS ACROSS CULTURES IN FACE AND EMOTION [J].
EKMAN, P ;
FRIESEN, WV .
JOURNAL OF PERSONALITY AND SOCIAL PSYCHOLOGY, 1971, 17 (02) :124-&
[10]   Unsupervised emotional state classification through physiological parameters for social robotics applications [J].
Fiorini, Laura ;
Mancioppi, Gianmaria ;
Semeraro, Francesco ;
Fujita, Hamido ;
Cavallo, Filippo .
KNOWLEDGE-BASED SYSTEMS, 2020, 190