CHANGE DETECTION, MULTIPLE CONTROLLERS, AND DYNAMIC ENVIRONMENTS: INSIGHTS FROM THE BRAIN

被引:8
作者
Pearson, John M. [1 ,2 ,3 ]
Platt, Michael L. [1 ,2 ,4 ]
机构
[1] Duke Univ, Med Ctr, Dept Neurobiol, Durham, NC 27706 USA
[2] Duke Univ, Ctr Cognit Neurosci, Durham, NC 27706 USA
[3] Duke Univ, Div Neurosurg, Med Ctr, Durham, NC 27706 USA
[4] Duke Univ, Dept Evolutionary Anthropol, Durham, NC 27706 USA
关键词
NEGATIVE REWARD SIGNALS; DECISION-MAKING; ORBITOFRONTAL CORTEX; FRONTOPOLAR CORTEX; PREFRONTAL CORTEX; LATERAL HABENULA; DEFAULT NETWORK; BASAL GANGLIA; DOPAMINE; NEURONS;
D O I
10.1002/jeab.5
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
Foundational studies in decision making focused on behavior as the most accessible and reliable data on which to build theories of choice. More recent work, however, has incorporated neural data to provide insights unavailable from behavior alone. Among other contributions, these studies have validated reinforcement learning models by demonstrating neural signals posited on the basis of behavioral work in classical and operant conditioning. In such models, the values of actions or options are updated incrementally based on the difference between expectations and outcomes, resulting in the gradual acquisition of stable behavior. By contrast, natural environments are often dynamic, including sudden, unsignaled shifts in reinforcement contingencies. Such rapid changes may necessitate frequent shifts in behavioral mode, requiring dynamic sensitivity to environmental changes. Recently, we proposed a model in which cingulate cortex plays a key role in detecting behaviorally relevant environmental changes and facilitating the update of multiple behavioral strategies. Here, we connect this framework to a model developed to handle the analogous problem in motor control. We offer a tentative dictionary of control signals in terms of brain structures and highlight key differences between motor and decision systems that may be important in evaluating the model.
引用
收藏
页码:74 / 84
页数:11
相关论文
共 73 条
[1]   Optogenetic Interrogation of Dopaminergic Modulation of the Multiple Phases of Reward-Seeking Behavior [J].
Adamantidis, Antoine R. ;
Tsai, Hsing-Chen ;
Boutrel, Benjamin ;
Zhang, Feng ;
Stuber, Garret D. ;
Budygin, Evgeny A. ;
Tourino, Clara ;
Bonci, Antonello ;
Deisseroth, Karl ;
de Lecea, Luis .
JOURNAL OF NEUROSCIENCE, 2011, 31 (30) :10829-10835
[2]   Reward encoding in the monkey anterior cingulate cortex [J].
Amiez, C. ;
Joseph, J. P. ;
Procyk, E. .
CEREBRAL CORTEX, 2006, 16 (07) :1040-1055
[3]  
[Anonymous], ARXIV07103742
[4]  
[Anonymous], 2002, BOUNDED RATIONALITY
[5]   The integrative function of the basal ganglia in instrumental conditioning [J].
Balleine, Bernard W. ;
Lijeholm, Mimi ;
Ostlund, Sean B. .
BEHAVIOURAL BRAIN RESEARCH, 2009, 199 (01) :43-52
[6]   Prefrontal cortex and decision making in a mixed-strategy game [J].
Barraclough, DJ ;
Conroy, ML ;
Lee, D .
NATURE NEUROSCIENCE, 2004, 7 (04) :404-410
[7]   Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions [J].
Bechara, A ;
Tranel, D ;
Damasio, H .
BRAIN, 2000, 123 :2189-2202
[8]   Learning the value of information in an uncertain world [J].
Behrens, Timothy E. J. ;
Woolrich, Mark W. ;
Walton, Mark E. ;
Rushworth, Matthew F. S. .
NATURE NEUROSCIENCE, 2007, 10 (09) :1214-1221
[9]   How Green Is the Grass on the Other Side? Frontopolar Cortex and the Evidence in Favor of Alternative Courses of Action [J].
Boorman, Erie D. ;
Behrens, Timothy E. J. ;
Woolrich, Mark W. ;
Rushworth, Matthew F. S. .
NEURON, 2009, 62 (05) :733-743
[10]   Nash Equilibria in Multi-Agent Motor Interactions [J].
Braun, Daniel A. ;
Ortega, Pedro A. ;
Wolpert, Daniel M. .
PLOS COMPUTATIONAL BIOLOGY, 2009, 5 (08)