Q-learning based Path Planning Method for UAVs using Priority Shifting

被引:14
作者
de Carvalho, Kevin B. [1 ]
de Oliveira, Iure Rosa L. [1 ]
Villa, Daniel K. D. [2 ]
Caldeira, Alexandre G. [1 ]
Sarcinelli-Filho, Mario [2 ]
Brandao, Alexandre S. [1 ,2 ]
机构
[1] Univ Fed Vicosa, Dept Elect Engn, Grad Program Comp Sci, Nucleus Specializat Robot, BR-36570900 Vicosa, MG, Brazil
[2] Univ Fed Espirito Santo, Dept Elect Engn, Grad Program Elect Engn, Vitoria, ES, Brazil
来源
2022 INTERNATIONAL CONFERENCE ON UNMANNED AIRCRAFT SYSTEMS (ICUAS) | 2022年
关键词
Mobile Robotics; Path Planning; Reinforcement Learning; Q-Learning;
D O I
10.1109/ICUAS54217.2022.9836175
中图分类号
V [航空、航天];
学科分类号
08 ; 0825 ;
摘要
Path planning is a crucial part of autonomous navigation when regarding autonomous aerial vehicles, often demanding different priorities such as the length, safety or energy consumption. Dynamic programming and geometric methods have been applied to solve this problem, but in recent years, more work has been developed using artificial intelligence approaches, such as reinforcement learning. In this paper we propose an offline path planning method for static environments using Q-learning. An optimal policy is found weighting three important factors: path length, safety and energy consumption. Due to a well balanced exploring/exploiting ratio, the proposed method can lead the agent to the desired destination starting from anywhere in the map. Simulations are done in different scenarios to address the performance of the proposed method and it showcased that the algorithm is able to find feasible paths in each scenario while regarding different set of priorities.
引用
收藏
页码:421 / 426
页数:6
相关论文
共 29 条
[1]   VBII-UAV: Vision-Based Infrastructure Inspection-UAV [J].
Al-Kaff, Abdulla ;
Miguel Moreno, Francisco ;
San Jose, Luis Javier ;
Garcia, Fernando ;
Martin, David ;
de la Escalera, Arturo ;
Nieva, Alberto ;
Meana Garcea, Jose Luis .
RECENT ADVANCES IN INFORMATION SYSTEMS AND TECHNOLOGIES, VOL 2, 2017, 570 :221-231
[2]   Artificial Intelligence in Advanced Manufacturing: Current Status and Future Outlook [J].
Arinez, Jorge F. ;
Chang, Qing ;
Gao, Robert X. ;
Xu, Chengying ;
Zhang, Jianjing .
JOURNAL OF MANUFACTURING SCIENCE AND ENGINEERING-TRANSACTIONS OF THE ASME, 2020, 142 (11)
[3]   A Path-Following Controller for a UAV-UGV Formation Performing the Final Step of Last-Mile-Delivery [J].
Bacheti, Vinicius Pacheco ;
Brandao, Alexandre Santos ;
Sarcinelli-Filho, Mario .
IEEE ACCESS, 2021, 9 :142218-142231
[4]  
Bohn E, 2019, INT CONF UNMAN AIRCR, P523, DOI [10.1109/ICUAS.2019.8798254, 10.1109/icuas.2019.8798254]
[5]  
Chakraborty Arpita, 2013, International Journal of Intelligent Mechatronics and Robotics, V3, P53, DOI 10.4018/ijimr.2013010105
[6]   Gestures-teleoperation of a heterogeneous multi-robot system [J].
de Carvalho, Kevin Braathen ;
Villa, Daniel Khede Dourado ;
Sarcinelli-Filho, Mario ;
Brandao, Alexandre Santos .
INTERNATIONAL JOURNAL OF ADVANCED MANUFACTURING TECHNOLOGY, 2022, 118 (5-6) :1999-2015
[7]  
Ding LG, 2018, INT C PAR DISTRIB SY, P339, DOI [10.1109/PADSW.2018.8644625, 10.1109/ICPADS.2018.00053]
[8]   Cooperative Load Transportation With Two Quadrotors Using Adaptive Control [J].
Dourado Villa, Daniel Khede ;
Brandao, Alexandre Santos ;
Carelli, Ricardo ;
Sarcinelli-Filho, Mario .
IEEE ACCESS, 2021, 9 :129148-129160
[9]   The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking [J].
Du, Dawei ;
Qi, Yuankai ;
Yu, Hongyang ;
Yang, Yifan ;
Duan, Kaiwen ;
Li, Guorong ;
Zhang, Weigang ;
Huang, Qingming ;
Tian, Qi .
COMPUTER VISION - ECCV 2018, PT X, 2018, 11214 :375-391
[10]  
Erdelj M, 2017, IEEE PERVAS COMPUT, V16, P24, DOI 10.1109/MPRV.2017.11