Value signals guide abstraction during learning

被引:16
作者
Cortese, Aurelio [1 ,2 ]
Yamamoto, Asuka [1 ,3 ]
Hashemzadeh, Maryam [4 ]
Sepulveda, Pradyumna [2 ]
Kawato, Mitsuo [1 ,5 ]
De Martino, Benedetto [2 ]
机构
[1] ATR Inst Int, Computat Neurosci Labs, Kyoto, Japan
[2] UCL, Inst Cognit Neurosci, London, England
[3] Nara Inst Sci & Technol, Sch Informat Sci, Nara, Japan
[4] Univ Alberta, Dept Comp Sci, Edmonton, AB, Canada
[5] RIKEN, Ctr Artificial Intelligence Project, Kyoto, Japan
基金
英国惠康基金;
关键词
DECODED FMRI NEUROFEEDBACK; ORBITOFRONTAL CORTEX; CONCEPTUAL KNOWLEDGE; COGNITIVE MAP; MECHANISMS; ATTENTION; REPRESENTATIONS; ARCHITECTURE; HIPPOCAMPUS; INTEGRATION;
D O I
10.7554/eLife.68943
中图分类号
Q [生物科学];
学科分类号
07 ; 0710 ; 09 ;
摘要
The human brain excels at constructing and using abstractions, such as rules, or concepts. Here, in two fMRI experiments, we demonstrate a mechanism of abstraction built upon the valuation of sensory features. Human volunteers learned novel association rules based on simple visual features. Reinforcement-learning algorithms revealed that, with learning, high-value abstract representations increasingly guided participant behaviour, resulting in better choices and higher subjective confidence. We also found that the brain area computing value signals - the ventromedial prefrontal cortex - prioritised and selected latent task elements during abstraction, both locally and through its connection to the visual cortex. Such a coding scheme predicts a causal role for valuation. Hence, in a second experiment, we used multivoxel neural reinforcement to test for the causality of feature valuation in the sensory cortex, as a mechanism of abstraction. Tagging the neural representation of a task feature with rewards evoked abstraction-based decisions. Together, these findings provide a novel interpretation of value as a goal-dependent, key factor in forging abstract representations.
引用
收藏
页数:27
相关论文
共 88 条
[31]   Exploiting Generalization in the Subspaces for Faster Model-Based Reinforcement Learning [J].
Hashemzadeh, Maryam ;
Hosseini, Reshad ;
Ahmadabadi, Majid Nili .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2019, 30 (06) :1635-1650
[32]   An empirical solution for over-pruning with a novel ensemble-learning method for fMRI decoding [J].
Hirose, Satoshi ;
Nambu, Isao ;
Naito, Eiichi .
JOURNAL OF NEUROSCIENCE METHODS, 2015, 239 :238-245
[33]   The value of abstraction [J].
Ho, Mark K. ;
Abel, David ;
Griffiths, Thomas L. ;
Littman, Michael L. .
CURRENT OPINION IN BEHAVIORAL SCIENCES, 2019, 29 :111-116
[34]   Adaptive Mixtures of Local Experts [J].
Jacobs, Robert A. ;
Jordan, Michael I. ;
Nowlan, Steven J. ;
Hinton, Geoffrey E. .
NEURAL COMPUTATION, 1991, 3 (01) :79-87
[35]   Decoding the visual and subjective contents of the human brain [J].
Kamitani, Y ;
Tong, F .
NATURE NEUROSCIENCE, 2005, 8 (05) :679-685
[36]   Efficient reinforcement learning: computational theories, neuroscience and robotics [J].
Kawato, Mitsuo ;
Samejima, Kazuyuki .
CURRENT OPINION IN NEUROBIOLOGY, 2007, 17 (02) :205-212
[37]   Constant Sub-second Cycling between Representations of Possible Futures in the Hippocampus [J].
Kay, Kenneth ;
Chung, Jason E. ;
Sosa, Marielena ;
Schor, Jonathan S. ;
Karlsson, Mattias P. ;
Larkin, Margaret C. ;
Liu, Daniel F. ;
Frank, Loren M. .
CELL, 2020, 180 (03) :552-+
[38]   Distributed neural representation of expected value [J].
Knutson, B ;
Taylor, J ;
Kaufman, M ;
Peterson, R ;
Glover, G .
JOURNAL OF NEUROSCIENCE, 2005, 25 (19) :4806-4812
[39]   Common neural code for reward and information value [J].
Kobayashi, Kenji ;
Hsu, Ming .
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2019, 116 (26) :13061-13066
[40]   Fear reduction without fear through reinforcement of neural activity that bypasses conscious exposure [J].
Koizumi, Ai ;
Amano, Kaoru ;
Cortese, Aurelio ;
Shibata, Kazuhisa ;
Yoshida, Wako ;
Seymour, Ben ;
Kawato, Mitsuo ;
Lau, Hakwan .
NATURE HUMAN BEHAVIOUR, 2017, 1 (01)