Improved classification performance of EEG-fNIRS multimodal brain-computer interface based on multi-domain features and multi-level progressive learning

被引:11
作者
Qiu, Lina [1 ]
Zhong, Yongshi [1 ]
He, Zhipeng [1 ]
Pan, Jiahui [1 ]
机构
[1] South China Normal Univ, Sch Software, Guangzhou, Peoples R China
来源
FRONTIERS IN HUMAN NEUROSCIENCE | 2022年 / 16卷
关键词
multimodal fusion; electroencephalogram (EEG); mental arithmetic (MA); functional near-infrared spectroscopy (fNIRS); multi-domain features; multi-level learning; motor imagery (MI); MOTOR IMAGERY; OPTIMIZATION; COMMUNICATION; SYSTEMS; FUSION; EMG;
D O I
10.3389/fnhum.2022.973959
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have potentially complementary characteristics that reflect the electrical and hemodynamic characteristics of neural responses, so EEG-fNIRS-based hybrid brain-computer interface (BCI) is the research hotspots in recent years. However, current studies lack a comprehensive systematic approach to properly fuse EEG and fNIRS data and exploit their complementary potential, which is critical for improving BCI performance. To address this issue, this study proposes a novel multimodal fusion framework based on multi-level progressive learning with multi-domain features. The framework consists of a multi-domain feature extraction process for EEG and fNIRS, a feature selection process based on atomic search optimization, and a multi-domain feature fusion process based on multi-level progressive machine learning. The proposed method was validated on EEG-fNIRS-based motor imagery (MI) and mental arithmetic (MA) tasks involving 29 subjects, and the experimental results show that multi-domain features provide better classification performance than single-domain features, and multi-modality provides better classification performance than single-modality. Furthermore, the experimental results and comparison with other methods demonstrated the effectiveness and superiority of the proposed method in EEG and fNIRS information fusion, it can achieve an average classification accuracy of 96.74% in the MI task and 98.42% in the MA task. Our proposed method may provide a general framework for future fusion processing of multimodal brain signals based on EEG-fNIRS.
引用
收藏
页数:16
相关论文
共 49 条
[1]   Stress Assessment Based on Decision Fusion of EEG and fNIRS Signals [J].
Al-Shargie, Fares ;
Tang, Tong Boon ;
Kiguchi, Masashi .
IEEE ACCESS, 2017, 5 :19889-19896
[2]  
Al-shargie F, 2016, 2016 6TH INTERNATIONAL CONFERENCE ON INTELLIGENT AND ADVANCED SYSTEMS (ICIAS)
[3]   An effective classification framework for brain-computer interface system design based on combining of fNIRS and EEG signals [J].
Alhudhaif, Adi .
PEERJ COMPUTER SCIENCE, 2021,
[4]  
Aris S., 2018, INT J ENG TECHNOL, V7, P44, DOI [10.14419/ijet.v7i4.11.20686, DOI 10.14419/IJET.V7I4.11.20686]
[5]   The Berlin Brain-Computer Interface: Progress Beyond Communication and Control [J].
Blankertz, Benjamin ;
Acqualagna, Laura ;
Dahne, Sven ;
Haufe, Stefan ;
-Kraft, Matthias Schultze ;
Sturm, Irene ;
Uscumlic, Marija ;
Wenzel, Markus A. ;
Curio, Gabriel ;
Muller, Klaus -Robert .
FRONTIERS IN NEUROSCIENCE, 2016, 10
[6]   Combined EEG-fNIRS Decoding of Motor Attempt and Imagery for Brain Switch Control: An Offline Study in Patients With Tetraplegia [J].
Blokland, Yvonne ;
Spyrou, Loukianos ;
Thijssen, Dick ;
Eijsvogels, Thijs ;
Colier, Willy ;
Floor-Westerdijk, Marianne ;
Vlek, Rutger ;
Bruhn, Jorgen ;
Farquhar, Jason .
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 2014, 22 (02) :222-229
[7]   Enhancing Communication for People in Late-Stage ALS Using an fNIRS-Based BCI System [J].
Borgheai, Seyyed Bahram ;
McLinden, John ;
Zisk, Alyssa Hillary ;
Hosni, Sarah Ismail ;
Deligani, Roohollah Jafari ;
Abtahi, Mohammadreza ;
Mankodiya, Kunal ;
Shahriari, Yalda .
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 2020, 28 (05) :1198-1207
[8]   Hybrid EEG-fNIRS Asynchronous Brain-Computer Interface for Multiple Motor Tasks [J].
Buccino, Alessio Paolo ;
Keles, Hasan Onur ;
Omurtag, Ahmet .
PLOS ONE, 2016, 11 (01)
[9]  
Ciftci Koray, 2008, 2008 IEEE 16th Signal Processing, Communication and Applications Conference (SIU), DOI 10.1109/SIU.2008.4632703
[10]   On the suitability of near-infrared (NIR) systems for next-generation brain-computer interfaces [J].
Coyle, S ;
Ward, T ;
Markham, C ;
McDarby, G .
PHYSIOLOGICAL MEASUREMENT, 2004, 25 (04) :815-822