Learning reward timing in cortex through reward dependent expression of synaptic plasticity

被引:52
作者
Gavornik, Jeffrey P. [1 ,2 ]
Shuler, Marshall G. Hussain [3 ]
Loewenstein, Yonatan [4 ,5 ]
Bear, Mark F. [6 ]
Shouval, Harel Z. [1 ]
机构
[1] Univ Texas Houston, Sch Med, Dept Neurobiol & Anat, Houston, TX 77030 USA
[2] Univ Texas Austin, Dept Elect & Comp Engn, Austin, TX 78712 USA
[3] Johns Hopkins Univ, Dept Neurosci, Baltimore, MD 21205 USA
[4] Hebrew Univ Jerusalem, Dept Neurobiol, Dept Cognit Sci, IL-91904 Jerusalem, Israel
[5] Hebrew Univ Jerusalem, Interdisciplinary Ctr Neural Computat, IL-91904 Jerusalem, Israel
[6] MIT, Howard Hughes Med Inst, Picower Inst Learning & Memory, Dept Brain & Cognit Sci, Cambridge, MA 02139 USA
基金
美国国家科学基金会; 以色列科学基金会;
关键词
reinforcment learning; visual cortex; PRIMARY VISUAL-CORTEX; PERSISTENT ACTIVITY; WORKING-MEMORY; NEURONAL-ACTIVITY; DOPAMINE NEURONS; AUDITORY-CORTEX; TIME PERCEPTION; BASAL FOREBRAIN; CEREBRAL-CORTEX; RAT;
D O I
10.1073/pnas.0901835106
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
The ability to represent time is an essential component of cognition but its neural basis is unknown. Although extensively studied both behaviorally and electrophysiologically, a general theoretical framework describing the elementary neural mechanisms used by the brain to learn temporal representations is lacking. It is commonly believed that the underlying cellular mechanisms reside in high order cortical regions but recent studies show sustained neural activity in primary sensory cortices that can represent the timing of expected reward. Here, we show that local cortical networks can learn temporal representations through a simple framework predicated on reward dependent expression of synaptic plasticity. We assert that temporal representations are stored in the lateral synaptic connections between neurons and demonstrate that reward-modulated plasticity is sufficient to learn these representations. We implement our model numerically to explain reward-time learning in the primary visual cortex (V1), demonstrate experimental support, and suggest additional experimentally verifiable predictions.
引用
收藏
页码:6826 / 6831
页数:6
相关论文
共 57 条
[1]   Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex [J].
Amit, DJ ;
Brunel, N .
CEREBRAL CORTEX, 1997, 7 (03) :237-252
[2]   Irregular persistent activity induced by synaptic excitatory feedback [J].
Barbieri, Francesca ;
Brunel, Nicolas .
FRONTIERS IN COMPUTATIONAL NEUROSCIENCE, 2007, 1
[3]   MODULATION OF VISUAL CORTICAL PLASTICITY BY ACETYLCHOLINE AND NORADRENALINE [J].
BEAR, MF ;
SINGER, W .
NATURE, 1986, 320 (6058) :172-176
[4]   Movement of 'gating charge' is coupled to ligand binding in a G-protein-coupled receptor [J].
Ben-Chaim, Yair ;
Chanda, Baron ;
Dascal, Nathan ;
Bezanilla, Francisco ;
Parnas, Itzchak ;
Parnas, Hanna .
NATURE, 2006, 444 (7115) :106-109
[5]   SYNAPTIC BACKGROUND ACTIVITY INFLUENCES SPATIOTEMPORAL INTEGRATION IN SINGLE PYRAMIDAL CELLS [J].
BERNANDER, O ;
DOUGLAS, RJ ;
MARTIN, KAC ;
KOCH, C .
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 1991, 88 (24) :11569-11573
[6]   Basic mechanisms for graded persistent activity: discrete attractors, continuous attractors, and dynamic representations [J].
Brody, CD ;
Romo, R ;
Kepecs, A .
CURRENT OPINION IN NEUROBIOLOGY, 2003, 13 (02) :204-211
[7]   Extrastriate feedback to primary visual cortex in primates: a quantitative analysis of connectivity [J].
Budd, JML .
PROCEEDINGS OF THE ROYAL SOCIETY B-BIOLOGICAL SCIENCES, 1998, 265 (1400) :1037-1044
[8]   How do we tell time? [J].
Buonomano, DV ;
Karmarkar, UR .
NEUROSCIENTIST, 2002, 8 (01) :42-51
[9]  
Dayan P., 2001, Theoretical neuroscience: Computational and mathematical modeling of neural systems
[10]   MYOSIN DEPHOSPHORYLATION DURING RAPID RELAXATION OF HOG CAROTID-ARTERY SMOOTH-MUSCLE [J].
DRISKA, SP ;
STEIN, PG ;
PORTER, R .
AMERICAN JOURNAL OF PHYSIOLOGY, 1989, 256 (02) :C315-C321