The role of training variability for model-based and model-free learning of an arbitrary visuomotor mapping

被引:0
作者
Velazquez-Vargas, Carlos A. [1 ]
Daw, Nathaniel D. [1 ,2 ]
Taylor, Jordan A. [1 ,2 ]
机构
[1] Princeton Univ, Dept Psychol, Princeton, NJ 08544 USA
[2] Princeton Univ, Princeton Neurosci Inst, Princeton, NJ USA
基金
美国国家卫生研究院;
关键词
SENSORY PREDICTION; SCHEMA THEORY; MOTOR; MOVEMENT; DYNAMICS; ADAPTATION; EXPLICIT; IMPLICIT; REPRESENTATIONS; ACQUISITION;
D O I
10.1371/journal.pcbi.1012471
中图分类号
Q5 [生物化学];
学科分类号
071010 ; 081704 ;
摘要
A fundamental feature of the human brain is its capacity to learn novel motor skills. This capacity requires the formation of vastly different visuomotor mappings. Using a grid navigation task, we investigated whether training variability would enhance the flexible use of a visuomotor mapping (key-to-direction rule), leading to better generalization performance. Experiments 1 and 2 show that participants trained to move between multiple start-target pairs exhibited greater generalization to both distal and proximal targets compared to participants trained to move between a single pair. This finding suggests that limited variability can impair decisions even in simple tasks without planning. In addition, during the training phase, participants exposed to higher variability were more inclined to choose options that, counterintuitively, moved the cursor away from the target while minimizing its actual distance under the constrained mapping, suggesting a greater engagement in model-based computations. In Experiments 3 and 4, we showed that the limited generalization performance in participants trained with a single pair can be enhanced by a short period of variability introduced early in learning or by incorporating stochasticity into the visuomotor mapping. Our computational modeling analyses revealed that a hybrid model between model-free and model-based computations with different mixing weights for the training and generalization phases, best described participants' data. Importantly, the differences in the model-based weights between our experimental groups, paralleled the behavioral findings during training and generalization. Taken together, our results suggest that training variability enables the flexible use of the visuomotor mapping, potentially by preventing the consolidation of habits due to the continuous demand to change responses. The development of new motor skills often requires the learning of novel associations between actions and outcomes. These novel mappings can be flexible and generalize to new situations, or more local with narrow generalization, similar to stimulus-action associations. In a series of experiments using a navigation task, we showed that generalizable mappings are favored under a training variability regime, while local mappings with narrow generalization are developed in the absence of variability. Training variability was generated in our experiments either with multiple goals or with stochasticity in the action-outcome mapping, with both regimes leading to successful generalization. In addition, we showed that the benefits in generalization from training variability can be observed even when participants are subsequently exposed to no variability for a prolonged period of time. These results were best described by a mixture of model-free and model-based reinforcement learning algorithms, with different mixture weights for the training and generalization phases.
引用
收藏
页数:43
相关论文
共 100 条
  • [51] Throwing while looking through prisms .1. Focal olivocerebellar lesions impair adaptation
    Martin, TA
    Keating, JG
    Goodkin, HP
    Bastian, AJ
    Thach, WT
    [J]. BRAIN, 1996, 119 : 1183 - 1198
  • [52] Generalization of Dynamics Learning Across Changes in Movement Amplitude
    Mattar, Andrew A. G.
    Ostry, David J.
    [J]. JOURNAL OF NEUROPHYSIOLOGY, 2010, 104 (01) : 426 - 438
  • [53] Planning in the brain
    Mattar, Marcelo G.
    Lengyel, Mate
    [J]. NEURON, 2022, 110 (06) : 914 - 934
  • [54] MCCRACKEN HD, 1977, J MOTOR BEHAV, V9, P193, DOI 10.1080/00222895.1977.10735109
  • [55] Revisiting the Role of the Medial Temporal Lobe in Motor Learning
    McDougle, Samuel D.
    Wilterson, Sarah A.
    Turk-Browne, Nicholas B.
    Taylor, Jordan A.
    [J]. JOURNAL OF COGNITIVE NEUROSCIENCE, 2022, 34 (03) : 532 - 549
  • [56] Explicit and Implicit Processes Constitute the Fast and Slow Processes of Sensorimotor Learning
    McDougle, Samuel D.
    Bond, Krista M.
    Taylor, Jordan A.
    [J]. JOURNAL OF NEUROSCIENCE, 2015, 35 (26) : 9568 - 9579
  • [57] Forward models for physiological motor control
    Miall, RC
    Wolpert, DM
    [J]. NEURAL NETWORKS, 1996, 9 (08) : 1265 - 1279
  • [58] Habits Without Values
    Miller, Kevin J.
    Shenhav, Amitai
    Ludvig, Elliot A.
    [J]. PSYCHOLOGICAL REVIEW, 2019, 126 (02) : 292 - 311
  • [59] Miller KJ., 2018, BioRxiv, P461129
  • [60] ASSESSMENT OF THE RESCORLA-WAGNER MODEL
    MILLER, RR
    BARNET, RC
    GRAHAME, NJ
    [J]. PSYCHOLOGICAL BULLETIN, 1995, 117 (03) : 363 - 386