CSF-GTNet: A Novel Multi-Dimensional Feature Fusion Network Based on Convnext-GeLU-BiLSTM for EEG-Signals-Enabled Fatigue Driving Detection

被引:36
作者
Gao, Dongrui [1 ,2 ]
Li, Pengrui [1 ]
Wang, Manqing [1 ,2 ]
Liang, Yujie [1 ]
Liu, Shihong [1 ]
Zhou, Jiliu [1 ]
Wang, Lutao [1 ]
Zhang, Yongqing [1 ]
机构
[1] Chengdu Univ Informat Technol, Sch Comp Sci, Chengdu 610225, Peoples R China
[2] Univ Elect Sci & Technol China, Sch Life Sci & Technol, Chengdu 611731, Peoples R China
基金
中国国家自然科学基金;
关键词
Multi-dimensional; Gaussian time domain network; pure convolutional spatial-frequency domain network; cross-subject difference; brain fatigue detection; RECOGNITION;
D O I
10.1109/JBHI.2023.3240891
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Electroencephalography (EEG) signal has been recognized as an effective fatigue detection method, which can intuitively reflect the drivers' mental state. However, the research on multi-dimensional features in existing work could be much better. The instability and complexity of EEG signals will increase the difficulty of extracting data features. More importantly, most current work only treats deep learning models as classifiers. They ignored the features of different subjects learned by the model. Aiming at the above problems, this paper proposes a novel multi-dimensional feature fusion network, CSF-GTNet, based on time and space-frequency domains for fatigue detection. Specifically, it comprises Gaussian Time Domain Network (GTNet) and Pure Convolutional Spatial Frequency Domain Network (CSFNet). The experimental results show that the proposed method effectively distinguishes between alert and fatigue states. The accuracy rates are 85.16% and 81.48% on the self-made and SEED-VIG datasets, respectively, which are higher than the state-of-the-art methods. Moreover, we analyze the contribution of each brain region for fatigue detection through the brain topology map. In addition, we explore the changing trend of each frequency band and the significance between different subjects in the alert state and fatigue state through the heat map. Our research can provide new ideas in brain fatigue research and play a specific role in promoting the development of this field.
引用
收藏
页码:2558 / 2568
页数:11
相关论文
共 32 条
[1]   Driver Mental Fatigue Detection Based on Head Posture Using New Modified reLU-BiLSTM Deep Neural Network [J].
Ansari, Shahzeb ;
Naghdy, Fazel ;
Du, Haiping ;
Pahnwar, Yasmeen Naz .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2022, 23 (08) :10957-10969
[2]  
Britton J. W., 2016, Electroencephalography (EEG): An introductory text and atlas of normal and abnormal findings in adults, children, and infants, DOI DOI 10.5698/978-0-9979756
[3]   Driver Fatigue Detection Through Chaotic Entropy Analysis of Cortical Sources Obtained From Scalp EEG Signals [J].
Chaudhuri, Aritra ;
Routray, Aurobinda .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2020, 21 (01) :185-198
[4]   EEG-Based Cross-Subject Driver Drowsiness Recognition With an Interpretable Convolutional Neural Network [J].
Cui, Jian ;
Lan, Zirui ;
Sourina, Olga ;
Muller-Wittig, Wolfgang .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2023, 34 (10) :7921-7933
[5]   Rhythm-Dependent Multilayer Brain Network for the Detection of Driving Fatigue [J].
Dang, Weidong ;
Gao, Zhongke ;
Lv, Dongmei ;
Sun, Xinlin ;
Cheng, Chichao .
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, 2021, 25 (03) :693-700
[6]   An Efficient LSTM Network for Emotion Recognition From Multichannel EEG Signals [J].
Du, Xiaobing ;
Ma, Cuixia ;
Zhang, Guanhua ;
Li, Jinyao ;
Lai, Yu-Kun ;
Zhao, Guozhen ;
Deng, Xiaoming ;
Liu, Yong-Jin ;
Wang, Hongan .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2022, 13 (03) :1528-1540
[7]   Detection of Train Driver Fatigue and Distraction Based on Forehead EEG: A Time-Series Ensemble Learning Method [J].
Fan, Chaojie ;
Peng, Yong ;
Peng, Shuangling ;
Zhang, Honghao ;
Wu, Yuankai ;
Kwong, Sam .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2022, 23 (08) :13559-13569
[8]   EEG-Based Spatio-Temporal Convolutional Neural Network for Driver Fatigue Evaluation [J].
Gao, Zhongke ;
Wang, Xinmin ;
Yang, Yuxuan ;
Mu, Chaoxu ;
Cai, Qing ;
Dang, Weidong ;
Zuo, Siyang .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2019, 30 (09) :2755-2763
[9]   Intelligent system for drowsiness recognition based on ear canal electroencephalography with photoplethysmography and electrocardiography [J].
Hong, Seunghyeok ;
Kwon, Hyunbin ;
Choi, Sang Ho ;
Park, Kwang Suk .
INFORMATION SCIENCES, 2018, 453 :302-322
[10]   RF-DCM: Multi-Granularity Deep Convolutional Model Based on Feature Recalibration and Fusion for Driver Fatigue Detection [J].
Huang, Rui ;
Wang, Yan ;
Li, Zijian ;
Lei, Zeyu ;
Xu, Yufan .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2022, 23 (01) :630-640