Reinforcement Learning for Continuous Control: A Quantum Normalized Advantage Function Approach

被引:0
|
作者
Liu, Yaofu [1 ]
Xu, Chang [1 ]
Jin, Siyuan [2 ]
机构
[1] Hong Kong Univ Sci & Technol, Dept Phys, Hong Kong, Peoples R China
[2] Hong Kong Univ Sci & Technol, Dept Informat Syst, Hong Kong, Peoples R China
关键词
Quantum Computation; Parameterized Quantum Circuit; Reinforcement Learning; Continuous Action Space;
D O I
10.1109/QSW59989.2023.00020
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
In this study, we present a new approach to quantum reinforcement learning that can handle tasks with a range of continuous actions. Our method uses a quantum version of the classic normalized advantage function (QNAF), only needing the Q-value network created by a quantum neural network and avoiding any policy network. We implemented the method by TensorFlow framework. When tested against standard Gym benchmarks, QNAF outperforms classical NAF and prior quantum methods in terms of fewer adjustable parameters. Furthermore, it shows improved stability, reliably converging regardless of changes in initial random parameters.
引用
收藏
页码:83 / 87
页数:5
相关论文
共 50 条
  • [1] Continuous Deep Q-Learning in Optimal Control Problems: Normalized Advantage Functions Analysis
    Plaksin, Anton
    Martyanov, Stepan
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,
  • [2] Reinforcement learning with multimodal advantage function for accurate advantage estimation in robot learning
    Park, Jonghyeok
    Han, Soohee
    ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2023, 126
  • [3] Reinforcement Learning in Continuous Time and Space: A Stochastic Control Approach
    Wang, Haoran
    Zariphopoulou, Thaleia
    Zhou, Xun Yu
    JOURNAL OF MACHINE LEARNING RESEARCH, 2020, 21
  • [4] A practical Reinforcement Learning implementation approach for continuous process control
    Patel, Kalpesh M.
    COMPUTERS & CHEMICAL ENGINEERING, 2023, 174
  • [5] Reinforcement learning in continuous time and space: A stochastic control approach
    Wang, Haoran
    Zariphopoulou, Thaleia
    Zhou, Xun Yu
    Journal of Machine Learning Research, 2020, 21
  • [6] Approximating the value function for continuous space reinforcement learning in robot control
    Buck, S
    Beetz, M
    Schmitt, T
    2002 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS, VOLS 1-3, PROCEEDINGS, 2002, : 1062 - 1067
  • [7] Safe reinforcement learning: A control barrier function optimization approach
    Marvi, Zahra
    Kiumarsi, Bahare
    INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL, 2021, 31 (06) : 1923 - 1940
  • [8] Deep Reinforcement Learning With Discrete Normalized Advantage Functions for Resource Management in Network Slicing
    Qi, Chen
    Hua, Yuxiu
    Li, Rongpeng
    Zhao, Zhifeng
    Zhang, Honggang
    IEEE COMMUNICATIONS LETTERS, 2019, 23 (08) : 1337 - 1341
  • [9] Quantum reinforcement learning in continuous action space
    Wu, Shaojun
    Jin, Shan
    Wen, Dingding
    Han, Donghong
    Wang, Xiaoting
    QUANTUM, 2025, 9 : 1 - 18
  • [10] HIERARCHICAL REINFORCEMENT LEARNING WITH ADVANTAGE FUNCTION FOR ENTITY RELATION EXTRACTION
    Zhu, Xianchao
    Zhu, William
    Journal of Applied and Numerical Optimization, 2022, 4 (03): : 393 - 404