Value signals guide abstraction during learning

被引:16
作者
Cortese, Aurelio [1 ,2 ]
Yamamoto, Asuka [1 ,3 ]
Hashemzadeh, Maryam [4 ]
Sepulveda, Pradyumna [2 ]
Kawato, Mitsuo [1 ,5 ]
De Martino, Benedetto [2 ]
机构
[1] ATR Inst Int, Computat Neurosci Labs, Kyoto, Japan
[2] UCL, Inst Cognit Neurosci, London, England
[3] Nara Inst Sci & Technol, Sch Informat Sci, Nara, Japan
[4] Univ Alberta, Dept Comp Sci, Edmonton, AB, Canada
[5] RIKEN, Ctr Artificial Intelligence Project, Kyoto, Japan
基金
英国惠康基金;
关键词
DECODED FMRI NEUROFEEDBACK; ORBITOFRONTAL CORTEX; CONCEPTUAL KNOWLEDGE; COGNITIVE MAP; MECHANISMS; ATTENTION; REPRESENTATIONS; ARCHITECTURE; HIPPOCAMPUS; INTEGRATION;
D O I
10.7554/eLife.68943
中图分类号
Q [生物科学];
学科分类号
07 ; 0710 ; 09 ;
摘要
The human brain excels at constructing and using abstractions, such as rules, or concepts. Here, in two fMRI experiments, we demonstrate a mechanism of abstraction built upon the valuation of sensory features. Human volunteers learned novel association rules based on simple visual features. Reinforcement-learning algorithms revealed that, with learning, high-value abstract representations increasingly guided participant behaviour, resulting in better choices and higher subjective confidence. We also found that the brain area computing value signals - the ventromedial prefrontal cortex - prioritised and selected latent task elements during abstraction, both locally and through its connection to the visual cortex. Such a coding scheme predicts a causal role for valuation. Hence, in a second experiment, we used multivoxel neural reinforcement to test for the causality of feature valuation in the sensory cortex, as a mechanism of abstraction. Tagging the neural representation of a task feature with rewards evoked abstraction-based decisions. Together, these findings provide a novel interpretation of value as a goal-dependent, key factor in forging abstract representations.
引用
收藏
页数:27
相关论文
共 88 条
[11]   Abstract Memory Representations in the Ventromedial Prefrontal Cortex and Hippocampus Support Concept Generalization [J].
Bowman, Caitlin R. ;
Zeithamova, Dagmar .
JOURNAL OF NEUROSCIENCE, 2018, 38 (10) :2605-2614
[12]   Anterior cingulate cortex, error detection, and the online monitoring of performance [J].
Carter, CS ;
Braver, TS ;
Barch, DM ;
Botvinick, MM ;
Noll, D ;
Cohen, JD .
SCIENCE, 1998, 280 (5364) :747-749
[13]   How usefulness shapes neural representations during goal-directed behavior [J].
Castegnetti, G. ;
Zurita, M. ;
De Martino, B. .
SCIENCE ADVANCES, 2021, 7 (15)
[14]   Organizing conceptual knowledge in humans with a gridlike code [J].
Constantinescu, Alexandra O. ;
O'Reilly, Jill X. ;
Behrens, Timothy E. J. .
SCIENCE, 2016, 352 (6292) :1464-1468
[15]   Unconscious reinforcement learning of hidden brain states supported by confidence [J].
Cortese, Aurelio ;
Lau, Hakwan ;
Kawato, Mitsuo .
NATURE COMMUNICATIONS, 2020, 11 (01)
[16]   The neural and cognitive architecture for learning from a small sample [J].
Cortese, Aurelio ;
De Martino, Benedetto ;
Kawato, Mitsuo .
CURRENT OPINION IN NEUROBIOLOGY, 2019, 55 :133-141
[17]   Decoded fMRI neurofeedback can induce bidirectional confidence changes within single participants [J].
Cortese, Aurelio ;
Amano, Kaoru ;
Koizumi, Ai ;
Lau, Hakwan ;
Kawato, Mitsuo .
NEUROIMAGE, 2017, 149 :323-337
[18]   Multivoxel neurofeedback selectively modulates confidence without changing perceptual performance [J].
Cortese, Aurelio ;
Amano, Kaoru ;
Koizumi, Ai ;
Kawato, Mitsuo ;
Lau, Hakwan .
NATURE COMMUNICATIONS, 2016, 7
[19]   Confidence in value-based choice [J].
De Martino, Benedetto ;
Fleming, Stephen M. ;
Garrett, Neil ;
Dolan, Raymond J. .
NATURE NEUROSCIENCE, 2013, 16 (01) :105-U147
[20]   The Neuro-Computational Architecture of Value-Based Selection in the Human Brain [J].
Domenech, Philippe ;
Redoute, Jerome ;
Koechlin, Etienne ;
Dreher, Jean-Claude .
CEREBRAL CORTEX, 2018, 28 (02) :585-601