Quantum reinforcement learning during human decision-making

被引:68
|
作者
Li, Ji-An [1 ,2 ]
Dong, Daoyi [3 ]
Wei, Zhengde [4 ]
Liu, Ying [5 ]
Pan, Yu [6 ]
Nori, Franco [7 ,8 ]
Zhang, Xiaochu [1 ,9 ,10 ,11 ]
机构
[1] Univ Sci & Technol China, Hefei Natl Lab Phys Sci Microscale, Affiliated Hosp 1, Eye Ctr,Dept Ophthalmol,Sch Life Sci,Div Life Sci, Hefei, Peoples R China
[2] Univ Sci & Technol China, Sch Management, Dept Stat & Finance, Hefei, Peoples R China
[3] Univ New South Wales, Sch Engn & Informat Technol, Canberra, ACT, Australia
[4] Shanghai Jiao Tong Univ, Sch Med, Shanghai Mental Hlth Ctr, Shanghai Key Lab Psychot Disorders, Shanghai, Peoples R China
[5] Univ Sci & Technol China, Div Life Sci & Med, Affiliated Hosp 1, Hefei, Peoples R China
[6] Shanghai Int Studies Univ, Sch Business & Management, Key Lab Appl Brain & Cognit Sci, Shanghai, Peoples R China
[7] RIKEN, Cluster Pioneering Res, Theoret Quantum Phys Lab, Wako, Japan
[8] Univ Michigan, Dept Phys, Ann Arbor, MI 48109 USA
[9] Anhui Mental Hlth Ctr, Hefei Med Res Ctr Alcohol Addict, Hefei, Peoples R China
[10] Tianjin Normal Univ, Acad Psychol & Behav, Tianjin, Peoples R China
[11] Univ Sci & Technol China, Ctr Biomed Engn, Hefei, Peoples R China
基金
澳大利亚研究理事会; 日本学术振兴会; 中国国家自然科学基金; 日本科学技术振兴机构;
关键词
NEURAL REPRESENTATION; UNCERTAINTY; MODELS; CORTEX; DISTINCT; REWARD; RISK; IMPLEMENTATION; SUPPLEMENTARY; INTERFERENCE;
D O I
10.1038/s41562-019-0804-2
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
Li et al. show that human value-based decision-making can be modelled using the quantum reinforcement learning framework. These new models reveal the importance of the medial frontal cortex in this quantum-like decision-making process. Classical reinforcement learning (CRL) has been widely applied in neuroscience and psychology; however, quantum reinforcement learning (QRL), which shows superior performance in computer simulations, has never been empirically tested on human decision-making. Moreover, all current successful quantum models for human cognition lack connections to neuroscience. Here we studied whether QRL can properly explain value-based decision-making. We compared 2 QRL and 12 CRL models by using behavioural and functional magnetic resonance imaging data from healthy and cigarette-smoking subjects performing the Iowa Gambling Task. In all groups, the QRL models performed well when compared with the best CRL models and further revealed the representation of quantum-like internal-state-related variables in the medial frontal gyrus in both healthy subjects and smokers, suggesting that value-based decision-making can be illustrated by QRL at both the behavioural and neural levels.
引用
收藏
页码:294 / 307
页数:14
相关论文
共 50 条
  • [1] Quantum reinforcement learning during human decision-making
    Ji-An Li
    Daoyi Dong
    Zhengde Wei
    Ying Liu
    Yu Pan
    Franco Nori
    Xiaochu Zhang
    Nature Human Behaviour, 2020, 4 : 294 - 307
  • [2] Reinforcement learning with hierarchical decision-making
    Cohen, Shahar
    Maimon, Oded
    Khmlenitsky, Evgeni
    ISDA 2006: SIXTH INTERNATIONAL CONFERENCE ON INTELLIGENT SYSTEMS DESIGN AND APPLICATIONS, VOL 3, 2006, : 177 - +
  • [3] Decision analysis and reinforcement learning in surgical decision-making
    Loftus, Tyler J.
    Filiberto, Amanda C.
    Li, Yanjun
    Balch, Jeremy
    Cook, Allyson C.
    Tighe, Patrick J.
    Efron, Philip A.
    Upchurch, Gilbert R., Jr.
    Rashidi, Parisa
    Li, Xiaolin
    Bihorac, Azra
    SURGERY, 2020, 168 (02) : 253 - 266
  • [4] REINFORCEMENT LEARNING FOR DECISION-MAKING IN A BUSINESS SIMULATOR
    Garcia, Javier
    Borrajo, Fernando
    Fernandez, Fernando
    INTERNATIONAL JOURNAL OF INFORMATION TECHNOLOGY & DECISION MAKING, 2012, 11 (05) : 935 - 960
  • [5] Transformer in reinforcement learning for decision-making: a survey
    Yuan, Weilin
    Chen, Jiaxing
    Chen, Shaofei
    Feng, Dawei
    Hu, Zhenzhen
    Li, Peng
    Zhao, Weiwei
    FRONTIERS OF INFORMATION TECHNOLOGY & ELECTRONIC ENGINEERING, 2024, 25 (06) : 763 - 790
  • [6] Quantum dynamics of human decision-making
    Busemeyer, Jerome R.
    Wang, Zheng
    Townsend, James T.
    JOURNAL OF MATHEMATICAL PSYCHOLOGY, 2006, 50 (03) : 220 - 241
  • [7] Invited: Efficient Reinforcement Learning for Automating Human Decision-Making in SoC Design
    Sadasivam, Shankar
    Chen, Zhuo
    Lee, Jinwon
    Jain, Rajeev
    2018 55TH ACM/ESDA/IEEE DESIGN AUTOMATION CONFERENCE (DAC), 2018,
  • [8] Reinforcement learning for decision-making under deep uncertainty
    Pei, Zhihao
    Rojas-Arevalo, Angela M.
    de Haan, Fjalar J.
    Lipovetzky, Nir
    Moallemi, Enayat A.
    JOURNAL OF ENVIRONMENTAL MANAGEMENT, 2024, 359
  • [9] Application of Reinforcement Learning in Multiagent Intelligent Decision-Making
    Han, Xiaoyu
    COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE, 2022, 2022
  • [10] Thinking as human: Self-reflective reinforcement learning framework for fertilization decision-making
    Li, Shulang
    Liu, Jie
    Yang, Yang
    Shen, Rujia
    Jiang, Jingchi
    SMART AGRICULTURAL TECHNOLOGY, 2025, 10