Autonomous reinforcement learning with experience replay

被引：38

作者：

Wawrzynski, Pawel ^{[1
]}

Tanwani, Ajay Kumar ^{[1
,2
]}

机构：

[1] Warsaw Univ Technol, Inst Control & Computat Engn, Warsaw, Poland

[2] Ecole Polytech Fed Lausanne, CH-1015 Lausanne, Switzerland

来源：

NEURAL NETWORKS | 2013年 / 41卷

关键词：

Reinforcement learning; Autonomous learning; Step-size estimation; Actor-critic; ACTOR-CRITIC ALGORITHMS; RATE ADAPTATION; ENVIRONMENTS; CONVERGENCE; NETWORKS;

D O I：

10.1016/j.neunet.2012.11.007

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

This paper considers the issues of efficiency and autonomy that are required to make reinforcement learning suitable for real-life control tasks. A real-time reinforcement learning algorithm is presented that repeatedly adjusts the control policy with the use of previously collected samples, and autonomously estimates the appropriate step-sizes for the learning updates. The algorithm is based on the actor-critic with experience replay whose step-sizes are determined on-line by an enhanced fixed point algorithm for on-line neural network training. An experimental study with simulated octopus arm and half-cheetah demonstrates the feasibility of the proposed algorithm to solve difficult learning control problems in an autonomous way within reasonably short time. (c) 2012 Elsevier Ltd. All rights reserved.

引用

页码：156 / 167

页数：12

共 50 条

[1] Autonomous Reinforcement Learning with Experience Replay for Humanoid Gait Optimization
Wawrzynski, Pawel
PROCEEDINGS OF THE INTERNATIONAL NEURAL NETWORK SOCIETY WINTER CONFERENCE (INNS-WC2012), 2012, 13 : 205 - 211
[2] Deep Reinforcement Learning for Autonomous Driving Based on Safety Experience Replay
Huang, Xiaohan
Cheng, Yuhu
Yu, Qiang
Wang, Xuesong
IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS, 2024, 16 (06) : 2070 - 2084
[3] Continuous Reinforcement Learning From Human Demonstrations With Integrated Experience Replay for Autonomous Driving
Zuo, Sixiang
Wang, Zhiyang
Zhu, Xiaorui
Ou, Yongsheng
2017 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (IEEE ROBIO 2017), 2017, : 2450 - 2455
[4] Deep Reinforcement Learning Based on the Hindsight Experience Replay for Autonomous Driving of Mobile Robot
Park M.
Hong J.S.
Kwon N.K.
Journal of Institute of Control, Robotics and Systems, 2022, 28 (11): : 1006 - 1012
[5] SELECTIVE EXPERIENCE REPLAY IN REINFORCEMENT LEARNING FOR REIDENTIFICATION
Thakoor, Ninad
Bhanu, Bir
2016 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2016, : 4250 - 4254
[6] Prioritized experience replay based reinforcement learning for adaptive tracking control of autonomous underwater vehicle
Li, Ting
Yang, Dongsheng
Xie, Xiangpeng
APPLIED MATHEMATICS AND COMPUTATION, 2023, 443
[7] Efficient experience replay architecture for offline reinforcement learning
Zhang, Longfei
Feng, Yanghe
Wang, Rongxiao
Xu, Yue
Xu, Naifu
Liu, Zeyi
Du, Hang
ROBOTIC INTELLIGENCE AND AUTOMATION, 2023, 43 (01): : 35 - 43
[8] Clustering experience replay for the effective exploitation in reinforcement learning
Li, Min
Huang, Tianyi
Zhu, William
PATTERN RECOGNITION, 2022, 131
[9] Deep Reinforcement Learning with Experience Replay Based on SARSA
Zhao, Dongbin
Wang, Haitao
Shao, Kun
Zhu, Yuanheng
PROCEEDINGS OF 2016 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (SSCI), 2016,
[10] Multimodal fusion for autonomous navigation via deep reinforcement learning with sparse rewards and hindsight experience replay
Xiao, Wendong
Yuan, Liang
Ran, Teng
He, Li
Zhang, Jianbo
Cui, Jianping
DISPLAYS, 2023, 78

← 1 2 3 4 5 →