Functional neural interactions during adaptive reward learning: An functional magnetic resonance imaging study

被引:3
作者
Wang, Ting [1 ]
Wu, Xi [1 ]
Jiang, Jiefeng [2 ]
Liu, Chang [3 ]
Zhu, Ming [4 ]
机构
[1] Chengdu Univ Informat Technol, Dept Comp Sci, Chengdu, Sichuan, Peoples R China
[2] Stanford Univ, Dept Psychol, Stanford, CA USA
[3] Chengdu Univ, Dept Informat Technol & Engn, Chengdu, Sichuan, Peoples R China
[4] Chengdu Univ Informat Technol, Dept Sci & Technol, 24 Block 1,Xue Fu Rd, Chengdu 610225, Sichuan, Peoples R China
基金
中国国家自然科学基金;
关键词
adaptive reward learning; functional magnetic resonance imaging; learning rate; prediction error; psychophysiological interaction analysis; MEDIAL FRONTAL-CORTEX; PREDICTION ERRORS; REINFORCEMENT; MECHANISMS; SIGNALS; MODEL; FMRI;
D O I
10.1002/ima.22387
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
A key feature of learning in humans is flexibility in adjusting the weight of new information to update predictions. This flexibility can be computationally captured by changing the learning rate in a reinforcement-learning model. Key components of reinforcement learning-such as prediction error (delta), learning rate (alpha), and reward feedback (r)-have been mapped to various brain areas. However, questions regarding the functional integration patterns in the human brain under the modulation of learning factors, and their interactions, remain unanswered. To investigate these phenomena, we first applied a reinforcement-learning model with an adaptive learning rate and functional magnetic resonance imaging to simulate the individual's reward-learning behavior. Psychophysiological interaction (PPI) analysis was then used to examine the functional interactions of the whole brain under the experimental condition of reward (r), the integration between reward and learning rate (alpha x r), and the integration between the prediction error and learning rate (alpha x delta) in a reward-learning task. The behavior statistical analyses indicated that the model estimates of alpha and delta captured the participants' learning behavioral patterns of getting high reward, by changing alpha and delta for different difficulties and after getting different reward feedback. The PPI analysis results showed that motor-related regions (including the supplement motor area, precentral gyrus, and thalamus) contributed to cognitive control processing regions (including the middle temporal gyrus, anterior and middle cingulate gyrus, and inferior frontal gyrus) by alpha x r. Finally, alpha x delta modulated the interaction between subregions of the striatum.
引用
收藏
页码:92 / 103
页数:12
相关论文
共 50 条
[41]   Neural correlates of anxiety sensitivity in panic disorder: A functional magnetic resonance imaging study [J].
Poletti, Sara ;
Radaelli, Daniele ;
Cucchi, Michele ;
Ricci, Liana ;
Vai, Benedetta ;
Smeraldi, Enrico ;
Benedetti, Francesco .
PSYCHIATRY RESEARCH-NEUROIMAGING, 2015, 233 (02) :95-101
[42]   Neural Substrates of Affective Face Recognition in Alexithymia: A Functional Magnetic Resonance Imaging Study [J].
Lee, Byeong-Taek ;
Lee, Hwa-Young ;
Park, Sae-Ah ;
Lim, Jin-Young ;
Tae, Woo Suk ;
Lee, Min-Soo ;
Joe, Sook-Haeng ;
Jung, In-Kwa ;
Ham, Byung-Joo .
NEUROPSYCHOBIOLOGY, 2011, 63 (02) :119-124
[43]   Simultaneous Electroencephalography and Functional Magnetic Resonance Imaging of General Anesthesia [J].
Purdon, Patrick L. ;
Pierce, Eric T. ;
Bonmassar, Giorgio ;
Walsh, John ;
Harrell, P. Grace ;
Kwo, Jean ;
Deschler, Daniel ;
Barlow, Margaret ;
Merhar, Rebecca C. ;
Lamus, Camilo ;
Mullaly, Catherine M. ;
Sullivan, Mary ;
Maginnis, Sharon ;
Skoniecki, Debra ;
Higgins, Helen-Anne ;
Brown, Emery N. .
DISORDERS OF CONSCIOUSNESS, 2009, 1157 :61-70
[44]   Pretherapeutic Functional Magnetic Resonance Imaging in Children [J].
Hertz-Pannier, Lucie ;
Noulhiane, Marion ;
Rodrigo, Sebastian ;
Chiron, Catherine .
NEUROIMAGING CLINICS OF NORTH AMERICA, 2014, 24 (04) :639-+
[45]   A method for functional magnetic resonance imaging of olfaction [J].
Sobel, N ;
Prabhakaran, V ;
Desmond, JE ;
Glover, GH ;
Sullivan, EV ;
Gabrieli, JDE .
JOURNAL OF NEUROSCIENCE METHODS, 1997, 78 (1-2) :115-123
[46]   Procedural learning in schizophrenia: a functional magnetic resonance imaging investigation [J].
Kumari, V ;
Gray, JA ;
Honey, GD ;
Soni, W ;
Bullmore, ET ;
Williams, SCR ;
Ng, VW ;
Vythelingum, GN ;
Simmons, A ;
Suckling, J ;
Corr, PJ ;
Sharma, T .
SCHIZOPHRENIA RESEARCH, 2002, 57 (01) :97-107
[47]   Procedural learning in schizophrenia investigated with functional magnetic resonance imaging [J].
Zedkova, Lenka ;
Woodward, Neil D. ;
Harding, Ian ;
Tibbo, Phil G. ;
Purdon, Scot E. .
SCHIZOPHRENIA RESEARCH, 2006, 88 (1-3) :198-207
[48]   Functional magnetic resonance imaging in the study of multiple sclerosis [J].
Labbe, Tomas ;
Ciampi, Ethel ;
Cruz, Juan P. ;
Zurita, Mariana ;
Uribe, Sergio ;
Carcamo, Claudia .
REVISTA DE NEUROLOGIA, 2018, 67 (03) :91-98
[49]   A functional magnetic resonance imaging study of episodic memory [J].
Busatto, G ;
Howard, RJ ;
Ha, Y ;
Brammer, M ;
Wright, I ;
Woodruff, PWR ;
Simmons, A ;
Williams, SCR ;
David, AS ;
Bullmore, ET .
NEUROREPORT, 1997, 8 (12) :2671-2675
[50]   Evidence of timing effects on acupuncture: A functional magnetic resonance imaging study [J].
Gao, Yanling ;
Lin, Zhicheng ;
Tao, Jing ;
Yang, Shanli ;
Chen, Ri ;
Jiang, Cai ;
Dong, Keng ;
Huang, Jia ;
Chen, Lidian .
EXPERIMENTAL AND THERAPEUTIC MEDICINE, 2015, 9 (01) :59-64