Dynamic Path Planning for Mobile Robots with Deep Reinforcement Learning

被引:16
作者
Yang, Laiyi [1 ]
Bi, Jing [1 ]
Yuan, Haitao [2 ]
机构
[1] Beijing Univ Technol, Sch Software Engn, Fac Informat Technol, Beijing 100124, Peoples R China
[2] Beihang Univ, Sch Automat Sci & Elect Engn, Beijing 100191, Peoples R China
来源
IFAC PAPERSONLINE | 2022年 / 55卷 / 11期
基金
中国国家自然科学基金;
关键词
Deep reinforcement learning; path planning; Soft Actor-Critic algorithm; continuous reward functions; mobile robots; ALGORITHM;
D O I
10.1016/j.ifacol.2022.08.042
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Traditional path planning algorithms for mobile robots are not effective to solve high-dimensional problems, and suffer from slow convergence and complex modelling. Therefore, it is highly essential to design a more efficient algorithm to realize intelligent path planning of mobile robots. This work proposes an improved path planning algorithm, which is based on the algorithm of Soft Actor-Critic (SAC). It attempts to solve a problem of poor robot performance in complicated environments with static and dynamic obstacles. This work designs an improved reward function to enable mobile robots to quickly avoid obstacles and reach targets by using state dynamic normalization and priority replay buffer techniques. To evaluate its performance, a Pygame-based simulation environment is constructed. The proposed method is compared with a Proximal Policy Optimization (PPO) algorithm in the simulation environment. Experimental results demonstrate that the cumulative reward of the proposed method is much higher than that of PPO, and it is also more robust than PPO. Copyright (C) 2022 The Authors.
引用
收藏
页码:19 / 24
页数:6
相关论文
共 18 条
  • [1] Akka K, 2018, INT J ADV ROBOT SYST, V15, P851
  • [2] A Voronoi-diagram-based dynamic path-planning system for underactuated marine vessels
    Candeloro, Mauro
    Lekkas, Anastasios M.
    Sorensen, Asgeir J.
    [J]. CONTROL ENGINEERING PRACTICE, 2017, 61 : 41 - 54
  • [3] A knowledge-free path planning approach for smart ships based on reinforcement learning
    Chen, Chen
    Chen, Xian-Qiao
    Ma, Feng
    Zeng, Xiao-Jun
    Wang, Jin
    [J]. OCEAN ENGINEERING, 2019, 189
  • [4] UAV path planning using artificial potential field method updated by optimal control theory
    Chen, Yong-bo
    Luo, Guan-chen
    Mei, Yue-song
    Yu, Jian-qiao
    Su, Xiao-long
    [J]. INTERNATIONAL JOURNAL OF SYSTEMS SCIENCE, 2016, 47 (06) : 1407 - 1420
  • [5] Intelligent-based multi-robot path planning inspired by improved classical Q-learning and improved particle swarm optimization with perturbed velocity
    Das, P. K.
    Behera, H. S.
    Panigrahi, B. K.
    [J]. ENGINEERING SCIENCE AND TECHNOLOGY-AN INTERNATIONAL JOURNAL-JESTECH, 2016, 19 (01): : 651 - 669
  • [6] Path planning with modified A star algorithm for a mobile robot
    Duchon, Frantisek
    Babinec, Andrej
    Kajan, Martin
    Beno, Peter
    Florek, Martin
    Fico, Tomas
    Jurisica, Ladislav
    [J]. MODELLING OF MECHANICAL AND MECHATRONIC SYSTEMS, 2014, 96 : 59 - 69
  • [7] Gasparetto A, 2015, MECH MACH SCI, V29, P3, DOI 10.1007/978-3-319-14705-5_1
  • [8] Harwin Laya, 2019, 2019 Third International Conference on Inventive Systems and Control (ICISC), P472, DOI 10.1109/ICISC44355.2019.9036354
  • [9] A Deterministic Improved Q-Learning for Path Planning of a Mobile Robot
    Konar, Amit
    Chakraborty, Indrani Goswami
    Singh, Sapam Jitu
    Jain, Lakhmi C.
    Nagar, Atulya K.
    [J]. IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2013, 43 (05): : 1141 - 1153
  • [10] Genetic Algorithm Based Approach for Autonomous Mobile Robot Path Planning
    Lamini, Chaymaa
    Benhlima, Said
    Elbekri, Ali
    [J]. PROCEEDINGS OF THE FIRST INTERNATIONAL CONFERENCE ON INTELLIGENT COMPUTING IN DATA SCIENCES (ICDS2017), 2018, 127 : 180 - 189