Decoding Visual fMRI Stimuli from Human Brain Based on Graph Convolutional Neural Network

被引:5
|
作者
Meng, Lu [1 ]
Ge, Kang [1 ]
机构
[1] Northeastern Univ, Coll Informat Sci & Engn, Shenyang 110000, Peoples R China
基金
中国国家自然科学基金;
关键词
brain decoding; convolutional neural network; graph convolution; functional magnetic resonance image; REPRESENTATION; MONKEY; AREA;
D O I
10.3390/brainsci12101394
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Brain decoding is to predict the external stimulus information from the collected brain response activities, and visual information is one of the most important sources of external stimulus information. Decoding functional magnetic resonance imaging (fMRI) based on visual stimulation is helpful in understanding the working mechanism of the brain visual function regions. Traditional brain decoding algorithms cannot accurately extract stimuli features from fMRI. To address these shortcomings, this paper proposed a brain decoding algorithm based on a graph convolution network (GCN). Firstly, 11 regions of interest (ROI) were selected according to the human brain visual function regions, which can avoid the noise interference of the non-visual regions of the human brain; then, a deep three-dimensional convolution neural network was specially designed to extract the features of these 11 regions; next, the GCN was used to extract the functional correlation features between the different human brain visual regions. Furthermore, to avoid the problem of gradient disappearance when there were too many layers of graph convolutional neural network, the residual connections were adopted in our algorithm, which helped to integrate different levels of features in order to improve the accuracy of the proposed GCN. The proposed algorithm was tested on the public dataset, and the recognition accuracy reached 98.67%. Compared with the other state-of-the-art algorithms, the proposed algorithm performed the best.
引用
收藏
页数:17
相关论文
共 50 条
  • [1] Decoding natural image stimuli from fMRI data with a surface-based convolutional network
    Gu, Zijin
    Jamison, Keith
    Kuceyeski, Amy
    Sabuncu, Mert
    MEDICAL IMAGING WITH DEEP LEARNING, VOL 227, 2023, 227 : 107 - 118
  • [2] Explainable fMRI-based brain decoding via spatial temporal-pyramid graph convolutional network
    Ye, Ziyuan
    Qu, Youzhi
    Liang, Zhichao
    Wang, Mo
    Liu, Quanying
    HUMAN BRAIN MAPPING, 2023, 44 (07) : 2921 - 2935
  • [3] Decoding of Emotional Visual Stimuli Using fMRI Brain Signal
    Yoshida, Shinichi
    2016 IEEE/ACIS 15TH INTERNATIONAL CONFERENCE ON COMPUTER AND INFORMATION SCIENCE (ICIS), 2016, : 925 - 928
  • [4] Decoding of visual activity patterns from fMRI responses using multivariate pattern analyses and convolutional neural network
    Zafar, Raheel
    Kamel, Nidal
    Naufal, Mohamad
    Malik, Aamir Saeed
    Dass, Sarat C.
    Ahmad, Rana Fayyaz
    Abdullah, Jafri M.
    Reza, Faruque
    JOURNAL OF INTEGRATIVE NEUROSCIENCE, 2017, 16 (03) : 275 - 289
  • [5] Decoding of Brain Signals to Detect Perceived Color-Stimuli using Convolutional Neural Network
    Laha, Mousumi
    Ghosh, Sayantani
    Bagchi, Anurag
    Pramanick, Shraman
    Konar, Amit
    2019 INTERNATIONAL CONFERENCE ON WIRELESS COMMUNICATIONS, SIGNAL PROCESSING AND NETWORKING (WISPNET 2019): ADVANCING WIRELESS AND MOBILE COMMUNICATIONS TECHNOLOGIES FOR 2020 INFORMATION SOCIETY, 2019, : 425 - 429
  • [6] Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices
    Qiao, Kai
    Chen, Jian
    Wang, Linyuan
    Zhang, Chi
    Zeng, Lei
    Tong, Li
    Yan, Bin
    FRONTIERS IN NEUROSCIENCE, 2019, 13
  • [7] fMRI-based Decoding of Visual Information from Human Brain Activity: A Brief Review
    Huang, Shuo
    Shao, Wei
    Wang, Mei-Ling
    Zhang, Dao-Qiang
    INTERNATIONAL JOURNAL OF AUTOMATION AND COMPUTING, 2021, 18 (02) : 170 - 184
  • [8] fMRI-based Decoding of Visual Information from Human Brain Activity: A Brief Review
    Shuo Huang
    Wei Shao
    Mei-Ling Wang
    Dao-Qiang Zhang
    International Journal of Automation and Computing, 2021, (02) : 170 - 184
  • [9] fMRI-based Decoding of Visual Information from Human Brain Activity: A Brief Review
    Shuo Huang
    Wei Shao
    Mei-Ling Wang
    Dao-Qiang Zhang
    International Journal of Automation and Computing, 2021, 18 : 170 - 184
  • [10] A neural decoding strategy based on convolutional neural network
    Hua, Shaoyang
    Wang, Congqing
    Wu, Xuewei
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2020, 39 (01) : 1033 - 1044