Model-Free Guidance Method for Drones in Complex Environments Using Direct Policy Exploration and Optimization

被引:3
|
作者
Liu, Hongxun [1 ]
Suzuki, Satoshi [1 ]
机构
[1] Chiba Univ, Sch Sci & Engn, 1-33 Yayoi Cho,Inage Ku, Chiba 2638522, Japan
关键词
drones; reinforcement learning; policy optimization; model-free; traverse complex environments;
D O I
10.3390/drones7080514
中图分类号
TP7 [遥感技术];
学科分类号
081102 ; 0816 ; 081602 ; 083002 ; 1404 ;
摘要
In the past few decades, drones have become lighter, with longer hang times, and exhibit more agile performance. To maximize their capabilities during flights in complex environments, researchers have proposed various model-based perception, planning, and control methods aimed at decomposing the problem into modules and collaboratively accomplishing the task in a sequential manner. However, in practical environments, it is extremely difficult to model both the drones and their environments, with very few existing model-based methods. In this study, we propose a novel model-free reinforcement-learning-based method that can learn the optimal planning and control policy from experienced flight data. During the training phase, the policy considers the complete state of the drones and environmental information as inputs. It then self-optimizes based on a predefined reward function. In practical implementations, the policy takes inputs from onboard and external sensors and outputs optimal control commands to low-level velocity controllers in an end-to-end manner. By capitalizing on this property, the planning and control policy can be improved without the need for an accurate system model and can drive drones to traverse complex environments at high speeds. The policy was trained and tested in a simulator, as well as in real-world flight experiments, demonstrating its practical applicability. The results show that this model-free method can learn to fly effectively and that it holds great potential to handle different tasks and environments.
引用
收藏
页数:19
相关论文
共 50 条
  • [31] Faded-Experience Trust Region Policy Optimization for Model-Free Power Allocation in Interference Channel
    Khoshkholgh, Mohammad G.
    Yanikomeroglu, Halim
    IEEE WIRELESS COMMUNICATIONS LETTERS, 2021, 10 (03) : 659 - 663
  • [32] A model-free voltage stability security assessment method using artificial intelligence
    Bastos, M. R.
    Martini, J. S. C.
    2015 IEEE PES INNOVATIVE SMART GRID TECHNOLOGIES LATIN AMERICA (ISGT LATAM), 2015, : 679 - 682
  • [33] A Model-free Mapless Navigation Method for Mobile Robot Using Reinforcement Learning
    Lv Qiang
    Duo Nanxun
    Lin Huican
    Wei Heng
    PROCEEDINGS OF THE 30TH CHINESE CONTROL AND DECISION CONFERENCE (2018 CCDC), 2018, : 3410 - 3415
  • [34] Model-free reinforcement learning for robust locomotion using demonstrations from trajectory optimization
    Bogdanovic, Miroslav
    Khadiv, Majid
    Righetti, Ludovic
    FRONTIERS IN ROBOTICS AND AI, 2022, 9
  • [35] Quality Control of Batch Processes Using Natural Gradient Based Model-Free Optimization
    Zhao, Fei
    Lu, Ningyun
    Lu, Jianhua
    INDUSTRIAL & ENGINEERING CHEMISTRY RESEARCH, 2014, 53 (44) : 17419 - 17428
  • [36] Analysis of electromagnetic response of cells and lipid membranes using a model-free method
    Lu, Yingxian
    Tang, Xiaping
    Zhao, Yanyu
    Jiang, Tianyu
    Zhou, Jiayao
    Wang, Xiaofei
    Huang, Bing
    Liu, Lingyu
    Deng, Hu
    Huang, Yujing
    Shi, Yigong
    BIOELECTROCHEMISTRY, 2023, 152
  • [37] A model-free and finite-time active disturbance rejection control method with parameter optimization
    Zhang, Zhen
    Guo, Yinan
    Zhu, Song
    Jiao, Feng
    Gong, Dunwei
    Song, Xianfang
    EXPERT SYSTEMS WITH APPLICATIONS, 2025, 278
  • [38] Thermal decomposition kinetics analysis of the oil sludge using model-based method and model-free method
    Liu, Hui
    Hong, Rui
    Xiang, Chenglang
    Wang, Haining
    Li, Yanqiang
    Xu, Guang
    Chang, Ping
    Zhu, Kai
    PROCESS SAFETY AND ENVIRONMENTAL PROTECTION, 2020, 141 : 167 - 177
  • [39] Model-Free Plug-n-Play Optimization Techniques to Design Autonomous and Resilient Complex Systems
    Vamvoudakis, K. G.
    Hespanha, J. P.
    2015 AMERICAN CONTROL CONFERENCE (ACC), 2015, : 5081 - 5081
  • [40] Model-Free Control Design for Loop Heat Pipes Using Deep Deterministic Policy Gradient
    Gellrich, Thomas
    Min, Yi
    Schwab, Stefan
    Hohmann, Soeren
    IFAC PAPERSONLINE, 2020, 53 (02): : 1575 - 1580