Spatiotemporal cortical dynamics for visual scene processing as revealed by EEG decoding

被引:0
作者
Orima, Taiki [1 ,2 ]
Motoyoshi, Isamu [1 ]
机构
[1] Univ Tokyo, Dept Life Sci, Tokyo, Japan
[2] Japan Soc Promot Sci, Tokyo, Japan
关键词
natural scene perception; EEG; brain decoding; EEGNet; Grad-CAM; TIME-COURSE; IMAGE STATISTICS; CATEGORIZATION; REPRESENTATION; PERCEPTION; AREA; MODULATION; STIMULI; CORTEX; SPEED;
D O I
10.3389/fnins.2023.1167719
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
The human visual system rapidly recognizes the categories and global properties of complex natural scenes. The present study investigated the spatiotemporal dynamics of neural signals involved in visual scene processing using electroencephalography (EEG) decoding. We recorded visual evoked potentials from 11 human observers for 232 natural scenes, each of which belonged to one of 13 natural scene categories (e.g., a bedroom or open country) and had three global properties (naturalness, openness, and roughness). We trained a deep convolutional classification model of the natural scene categories and global properties using EEGNet. Having confirmed that the model successfully classified natural scene categories and the three global properties, we applied Grad-CAM to the EEGNet model to visualize the EEG channels and time points that contributed to the classification. The analysis showed that EEG signals in the occipital electrodes at short latencies (approximately 80 similar to ms) contributed to the classifications, whereas those in the frontal electrodes at relatively long latencies (200 similar to ms) contributed to the classification of naturalness and the individual scene category. These results suggest that different global properties are encoded in different cortical areas and with different timings, and that the combination of the EEGNet model and Grad-CAM can be a tool to investigate both temporal and spatial distribution of natural scene processing in the human brain.
引用
收藏
页数:11
相关论文
共 73 条
[21]   How do amplitude spectra influence rapid animal detection? [J].
Gaspar, Carl M. ;
Rousselet, Guillaume A. .
VISION RESEARCH, 2009, 49 (24) :3001-3012
[22]  
Ghebreab Sennay., 2009, ADV NEURAL INFORM PR, P629
[23]   Disentangling the Independent Contributions of Visual and Conceptual Features to the Spatiotemporal Dynamics of Scene Categorization [J].
Greene, Michelle R. ;
Hansen, Bruce C. .
JOURNAL OF NEUROSCIENCE, 2020, 40 (27) :5283-5299
[24]   The Briefest of Glances: The Time Course of Natural Scene Understanding [J].
Greene, Michelle R. ;
Oliva, Aude .
PSYCHOLOGICAL SCIENCE, 2009, 20 (04) :464-472
[25]   Contributions of low- and high-level properties to neural processing of visual scenes in the human brain [J].
Groen, Iris I. A. ;
Silson, Edward H. ;
Baker, Chris I. .
PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY B-BIOLOGICAL SCIENCES, 2017, 372 (1714)
[26]   The time course of natural scene perception with reduced attention [J].
Groen, Iris I. A. ;
Ghebreab, Sennay ;
Lamme, Victor A. F. ;
Scholte, H. Steven .
JOURNAL OF NEUROPHYSIOLOGY, 2016, 115 (02) :931-946
[27]   From Image Statistics to Scene Gist: Evoked Neural Activity Reveals Transition from Low-Level Natural Image Structure to Scene Category [J].
Groen, Iris I. A. ;
Ghebreab, Sennay ;
Prins, Hielke ;
Lamme, Victor A. F. ;
Scholte, H. Steven .
JOURNAL OF NEUROSCIENCE, 2013, 33 (48) :18814-18824
[28]   Spatially Pooled Contrast Responses Predict Neural and Perceptual Similarity of Naturalistic Image Categories [J].
Groen, Iris I. A. ;
Ghebreab, Sennay ;
Lamme, Victor A. F. ;
Scholte, H. Steven .
PLOS COMPUTATIONAL BIOLOGY, 2012, 8 (10)
[29]   From spatial frequency contrast to edge preponderance: the differential modulation of early visual evoked potentials by natural scene stimuli [J].
Hansen, Bruce C. ;
Jacques, Theodore ;
Johnson, Aaron P. ;
Ellemberg, Dave .
VISUAL NEUROSCIENCE, 2011, 28 (03) :221-237
[30]   The influence of behavioral relevance on the processing of global scene properties: An ERP study [J].
Hansen, Natalie E. ;
Noesen, Birken T. ;
Nador, Jeffrey D. ;
Harel, Assaf .
NEUROPSYCHOLOGIA, 2018, 114 :168-180