A deep learning approach for decoding visually imagined digits and lettersusing time-frequency-spatial representation of EEG signals

被引:9
作者
Alazrai, Rami [1 ]
Abuhijleh, Motaz [1 ]
Ali, Mostafa Z. [2 ]
Daoud, Mohammad I. [1 ]
机构
[1] German Jordanian Univ, Sch Elect Engn & Informat Technol, Dept Comp Engn, Amman 11180, Jordan
[2] Jordan Univ Sci & Technol, Dept Comp Informat Syst, Irbid 22110, Jordan
关键词
Electroencephalography; Time-frequency-spatial representation; Visual imagery; Deep learning; Convolutional neural networks; Brain-computer interface; MOTOR IMAGERY; MENTAL-IMAGERY; NEURAL-NETWORK; CLASSIFICATION; AGREEMENT; FRAMEWORK; FMRI;
D O I
10.1016/j.eswa.2022.117417
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The recent advances in developing assistive devices have attracted researchers to use visual imagery (VI) mentaltasks as a control paradigm to design brain-computer interfaces that can produce a large number of controlsignals. Consequently, this can facilitate the design of control mechanisms that allow locked-in individuals tointeract with the surrounding world. This paper presents a two-phase approach for decoding visually imagineddigits and letters using electroencephalography (EEG) signals. The first phase employs the Choi-Williams time-frequency distribution (CWD) to construct a joint time, frequency, and spatial (TFS) representation of the EEGsignals. The constructed joint TFS representation characterizes the variations in the energy encapsulated withinthe EEG signals over the TFS domains. The second phase presents a novel deep learning (DL) framework toautomatically extract features from the constructed joint TFS representation of the EEG signals and decodethe imagined digits and letters. The performance of our approach is assessed using an EEG dataset that wasacquired for 16 healthy participants while imagining decimal digits and uppercase English letters. Our approachachieved an average +/- standard deviation accuracy of95.47 +/- 2.3%, which is significantly outperformingthe accuracies obtained when the CWD is replaced with two alternative time-frequency analysis techniques,the accuracies obtained using four pre-trained DL models, and the accuracies obtained using CWD-basedhandcrafted features that are classified using four conventional classifiers. Moreover, the results of our proposedapproach outperform those reported by several previous studies with regard to the accuracy and number ofclasses.
引用
收藏
页数:17
相关论文
共 67 条
[1]   Performance variation in motor imagery brain-computer interface: A brief review [J].
Ahn, Minkyu ;
Jun, Sung Chan .
JOURNAL OF NEUROSCIENCE METHODS, 2015, 243 :103-110
[2]   A Deep Learning Framework for Decoding Motor Imagery Tasks of the Same Hand Using EEG Signals [J].
Alazrai, Rami ;
Abuhijleh, Motaz ;
Alwanni, Hisham ;
Daoud, Mohammad, I .
IEEE ACCESS, 2019, 7 :109612-109627
[3]   Tonic Cold Pain Detection Using Choi-Williams Time-Frequency Distribution Analysis of EEG Signals: A Feasibility Study [J].
Alazrai, Rami ;
AL-Rawi, Saifaldeen ;
Alwanni, Hisham ;
Daoud, Mohammad I. .
APPLIED SCIENCES-BASEL, 2019, 9 (16)
[4]   EEG-based BCI system for decoding finger movements within the same hand [J].
Alazrai, Rami ;
Alwanni, Hisham ;
Daoud, Mohammad I. .
NEUROSCIENCE LETTERS, 2019, 698 :113-120
[5]   EEG-based tonic cold pain recognition system using wavelet transform [J].
Alazrai, Rami ;
Momani, Mohammad ;
Abu Khudair, Hussein ;
Daoud, Mohammad, I .
NEURAL COMPUTING & APPLICATIONS, 2019, 31 (07) :3187-3200
[6]   EEG-Based Emotion Recognition Using Quadratic Time-Frequency Distribution [J].
Alazrai, Rami ;
Homoud, Rasha ;
Alwanni, Hisham ;
Daoud, Mohammad I. .
SENSORS, 2018, 18 (08)
[7]   EEG-Based Brain-Computer Interface for Decoding Motor Imagery Tasks within the Same Hand Using Choi-Williams Time-Frequency Distribution [J].
Alazrai, Rami ;
Alwanni, Hisham ;
Baslan, Yara ;
Alnuman, Nasim ;
Daoud, Mohammad I. .
SENSORS, 2017, 17 (09)
[8]   Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex [J].
Albers, Anke Marit ;
Kok, Peter ;
Toni, Ivan ;
Dijkerman, H. Chris ;
de Lange, Floris P. .
CURRENT BIOLOGY, 2013, 23 (15) :1427-1431
[9]  
[Anonymous], 2009, Discrete quadratic time-frequency distributions: Definition, computation, and a newborn electroencephalogram application
[10]  
[Anonymous], 2018, Journal of neural engineering, DOI DOI 10.1088/1741-2552/AACE8C