Deep Reinforcement Learning for Drone Delivery

被引:44
|
作者
Munoz, Guillem [1 ]
Barrado, Cristina [1 ]
Cetin, Ender [1 ]
Salami, Esther [1 ]
机构
[1] UPC BarcelonaTECH, Comp Architecture Dept, Esteve Terrades 7, Castelldefels 08860, Spain
关键词
drones; deep learning; reinforcement learning; Q-learning; DQN; JNN;
D O I
10.3390/drones3030072
中图分类号
TP7 [遥感技术];
学科分类号
081102 ; 0816 ; 081602 ; 083002 ; 1404 ;
摘要
Drones are expected to be used extensively for delivery tasks in the future. In the absence of obstacles, satellite based navigation from departure to the geo-located destination is a simple task. When obstacles are known to be in the path, pilots must build a flight plan to avoid them. However, when they are unknown, there are too many or they are in places that are not fixed positions, then to build a safe flight plan becomes very challenging. Moreover, in a weak satellite signal environment, such as indoors, under trees canopy or in urban canyons, the current drone navigation systems may fail. Artificial intelligence, a research area with increasing activity, can be used to overcome such challenges. Initially focused on robots and now mostly applied to ground vehicles, artificial intelligence begins to be used also to train drones. Reinforcement learning is the branch of artificial intelligence able to train machines. The application of reinforcement learning to drones will provide them with more intelligence, eventually converting drones in fully-autonomous machines. In this work, reinforcement learning is studied for drone delivery. As sensors, the drone only has a stereo-vision front camera, from which depth information is obtained. The drone is trained to fly to a destination in a neighborhood environment that has plenty of obstacles such as trees, cables, cars and houses. The flying area is also delimited by a geo-fence; this is a virtual (non-visible) fence that prevents the drone from entering or leaving a defined area. The drone has to avoid visible obstacles and has to reach a goal. Results show that, in comparison with the previous results, the new algorithms have better results, not only with a better reward, but also with a reduction of its variance. The second contribution is the checkpoints. They consist of saving a trained model every time a better reward is achieved. Results show how checkpoints improve the test results.
引用
收藏
页码:1 / 19
页数:19
相关论文
共 50 条
  • [21] A deep reinforcement learning approach for solving the Traveling Salesman Problem with Drone
    Bogyrbayeva, Aigerim
    Yoon, Taehyun
    Ko, Hanbum
    Lim, Sungbin
    Yun, Hyokun
    Kwon, Changhyun
    TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES, 2023, 148
  • [22] Champion-level drone racing using deep reinforcement learning
    Elia Kaufmann
    Leonard Bauersfeld
    Antonio Loquercio
    Matthias Müller
    Vladlen Koltun
    Davide Scaramuzza
    Nature, 2023, 620 : 982 - 987
  • [23] Champion-level drone racing using deep reinforcement learning
    Kaufmann, Elia
    Bauersfeld, Leonard
    Loquercio, Antonio
    Mueller, Matthias
    Koltun, Vladlen
    Scaramuzza, Davide
    NATURE, 2023, 620 (7976) : 982 - +
  • [24] Drone patrolling with reinforcement learning
    Piciarelli, Claudio
    Foresti, Gian Luca
    ICDSC 2019: 13TH INTERNATIONAL CONFERENCE ON DISTRIBUTED SMART CAMERAS, 2019,
  • [25] Routing with Pickup and Delivery via Deep Reinforcement Learning
    Yildiz, Ozge Aslan
    Saricicek, Inci
    Ozkan, Kemal
    Yazici, Ahmet
    32ND IEEE SIGNAL PROCESSING AND COMMUNICATIONS APPLICATIONS CONFERENCE, SIU 2024, 2024,
  • [26] Counter a Drone and the Performance Analysis of Deep Reinforcement Learning Method and Human Pilot
    Cetin, Ender
    Barrado, Cristina
    Pastor, Enric
    2021 IEEE/AIAA 40TH DIGITAL AVIONICS SYSTEMS CONFERENCE (DASC), 2021,
  • [27] A deep reinforcement learning approach for the meal delivery problem
    Jahanshahi, Hadi
    Bozanta, Aysun
    Cevik, Mucahit
    Kavuk, Eray Mert
    Tosun, Ayse
    Sonuc, Sibel B.
    Kosucu, Bilgin
    Basar, Ayse
    KNOWLEDGE-BASED SYSTEMS, 2022, 243
  • [28] Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learning
    Hicks, Chris
    Mavroudis, Vasilios
    Foley, Myles
    Davies, Thomas
    Highnam, Kate
    Watson, Tim
    PROCEEDINGS OF THE 16TH ACM WORKSHOP ON ARTIFICIAL INTELLIGENCE AND SECURITY, AISEC 2023, 2023, : 91 - 101
  • [29] Obstacle Avoidance Drone by Deep Reinforcement Learning and Its Racing with Human Pilot
    Shin, Sang-Yun
    Kang, Yong-Won
    Kim, Yong-Guk
    APPLIED SCIENCES-BASEL, 2019, 9 (24):
  • [30] Autonomous multi-drone racing method based on deep reinforcement learning
    Kang, Yu
    Di, Jian
    Li, Ming
    Zhao, Yunbo
    Wang, Yuhui
    SCIENCE CHINA-INFORMATION SCIENCES, 2024, 67 (08)