ACE (Actor-Critic-Explorer) paradigm for reinforcement learning in basal ganglia Highlighting the role of subthalamic and pallidal nuclei

被引：7

作者：

Joseph, Denny ^{[1
]}

Gangadhar, Garipelli ^{[2
]}

Chakravarthy, V. Srinivasa ^{[1
]}

机构：

[1] Indian Inst Technol Madras, Dept Biotechnol, Madras 600036, Tamil Nadu, India

[2] IDIAP Res Inst, Machine Learning Grp, CH-1920 Martigny, Switzerland

来源：

NEUROCOMPUTING | 2010年 / 74卷 / 1-3期

关键词：

Basal ganglia; Dopamine; Norepinephrine; Reinforcement learning; Reaching; POSTERIOR PARIETAL CORTEX; DEEP BRAIN-STIMULATION; GLOBUS-PALLIDUS; PARKINSONS-DISEASE; REACHING MOVEMENTS; ERROR-CORRECTION; MPTP MODEL; DOPAMINE; REWARD; OSCILLATIONS;

D O I：

10.1016/j.neucom.2010.03.001

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We present a comprehensive model of basal ganglia in which the three important reinforcement learning components Actor Critic and Explorer (ACE) are represented and their anatomical substrates are identified Particularly we identify the subthalamic-nucleus and globus pallidus externa (STN-GPe) loop as the Explorer and argue that complex activity of STN and GPe neurons found in experimental studies provides the stochastic drive necessary for exploration Simulations involving a two-link arm model show task-dependent variations in complexity of STN-GPe activity when the ACE network is trained to perform simple reaching movements Complexity and average levels of STN-GPe activity are observed to be higher before training than in post-training conditions Further in order to simulate Parkinsonian conditions when dopamine levels in substantia nigra portion of the model are reduced the arm displayed as a primary change small amplitude movements which on persistent network training amplified to large amplitude unregulated movements reminiscent of Parkinsonian tremor (C) 2010 Elsevier B V All rights reserved

引用

页码：205 / 218

页数：14