A Study on Image Reconstruction Based on Decoding fMRI Through Extracting Image Depth Features

被引:0
|
作者
Deng, Xin [1 ]
Bao, Feiyang [1 ]
Liu, Bin [1 ]
Li, Yijia [1 ]
Zhang, Lianhua [1 ]
机构
[1] Chongqing Univ Posts & Telecommun, Chongqing 400065, Peoples R China
来源
NEURAL COMPUTING FOR ADVANCED APPLICATIONS, NCAA 2024, PT I | 2025年 / 2181卷
关键词
Image reconstruction; Neural decoding; Functional magnetic resonance imaging; Feature extraction; Visual Transformer; FACES;
D O I
10.1007/978-981-97-7001-4_32
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Visualizing perceived content through functional magnetic resonance imaging (fMRI) analysis is a captivating research area in brain decoding. Previous studies have primarily focused on restoring either high-level semantic features or low-level semantic features from fMRI data, but rarely achieved effective restoration of both. This study proposes a novel approach for decoding the visual cortex activity measured by fMRI into the layered visual features that share the hierarchical information from the corresponding images. By iteratively optimizing the relationship between the layered visual features and the image's depth features extracted by a visual transformer, the method in this research significantly improves the reconstruction of the image's deep features. Furthermore, by incorporating the prior natural image information through a deep generator network, this work enhances the reconstruction process, resulting in richer semantic details. Experimental results verify the effectiveness of our methodology in restoring both high-level and low-level semantic features of the perceived images, ultimately enhancing the overall visual fidelity of the reconstructed image. Importantly, our model demonstrates successful generalization to reconstruct artificial shapes, indicating that the performance of our model is not simply achieved by relying on extensive sample datasets. These findings prove the efficacy of the method in effectively reconstructing the perceived content based on the hierarchical neural representations, providing a new method to study the brain's underlying mechanisms.
引用
收藏
页码:449 / 462
页数:14
相关论文
共 50 条
  • [31] Depth Image Super-resolution Reconstruction Based on Filter Fusion
    He, Ying
    Liang, Bin
    Yang, Jun
    He, Jin
    Luan, Mengkai
    NINTH INTERNATIONAL CONFERENCE ON DIGITAL IMAGE PROCESSING (ICDIP 2017), 2017, 10420
  • [32] Super-Resolution Reconstruction of Depth Image Based on Kriging Interpolation
    Huang, Tingsheng
    Wang, Xinjian
    Wang, Chunyang
    Liu, Xuelian
    Yu, Yanqing
    APPLIED SCIENCES-BASEL, 2023, 13 (06):
  • [33] Image reconstruction of traditional Chinese painting works based on depth learning
    Li, Xiyang
    Ku, Wenzhen
    International Journal of Reasoning-based Intelligent Systems, 2024, 16 (04) : 267 - 277
  • [34] Contour reconstruction method for noisy image based on depth residual learning
    Wang X.
    Zhang S.
    Zhang J.
    Yuan S.
    Xi'an Dianzi Keji Daxue Xuebao/Journal of Xidian University, 2020, 47 (03): : 66 - 71
  • [35] Improved Upsampling Based Depth Image Super-Resolution Reconstruction
    Ye, Yanming
    Zhou, Mengxiong
    Wang, Zhanyu
    Shen, Xingfa
    IEEE ACCESS, 2023, 11 : 46782 - 46792
  • [36] Extracting depth information from microscopic image of micro manipulator
    Zhao, X
    Sun, MZ
    Yu, B
    Lu, GZ
    Liu, JT
    Huang, DG
    IEEE ROBIO 2004: Proceedings of the IEEE International Conference on Robotics and Biomimetics, 2004, : 629 - 633
  • [37] Learned Dynamic Guidance for Depth Image Reconstruction
    Gu, Shuhang
    Guo, Shi
    Zuo, Wangmeng
    Chen, Yunjin
    Timofte, Radu
    Van Gool, Luc
    Zhang, Lei
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2020, 42 (10) : 2437 - 2452
  • [38] Decoding natural image stimuli from fMRI data with a surface-based convolutional network
    Gu, Zijin
    Jamison, Keith
    Kuceyeski, Amy
    Sabuncu, Mert
    MEDICAL IMAGING WITH DEEP LEARNING, VOL 227, 2023, 227 : 107 - 118
  • [39] Decoding image semantics based on image background and salient objects
    Zhu, HY
    Sun, ZX
    Ma, C
    Zhou, RH
    COMPUTER SCIENCE AND TECHNOLOGY IN NEW CENTURY, 2001, : 218 - 222
  • [40] Research on features matching method for visual odometry based on depth image
    Wang Guosheng
    Guo Feng
    Lv Qiang
    Ma Jianye
    SIXTH INTERNATIONAL CONFERENCE ON ELECTRONICS AND INFORMATION ENGINEERING, 2015, 9794