Deep Reinforcement Learning for Drone Delivery

被引:44
|
作者
Munoz, Guillem [1 ]
Barrado, Cristina [1 ]
Cetin, Ender [1 ]
Salami, Esther [1 ]
机构
[1] UPC BarcelonaTECH, Comp Architecture Dept, Esteve Terrades 7, Castelldefels 08860, Spain
关键词
drones; deep learning; reinforcement learning; Q-learning; DQN; JNN;
D O I
10.3390/drones3030072
中图分类号
TP7 [遥感技术];
学科分类号
081102 ; 0816 ; 081602 ; 083002 ; 1404 ;
摘要
Drones are expected to be used extensively for delivery tasks in the future. In the absence of obstacles, satellite based navigation from departure to the geo-located destination is a simple task. When obstacles are known to be in the path, pilots must build a flight plan to avoid them. However, when they are unknown, there are too many or they are in places that are not fixed positions, then to build a safe flight plan becomes very challenging. Moreover, in a weak satellite signal environment, such as indoors, under trees canopy or in urban canyons, the current drone navigation systems may fail. Artificial intelligence, a research area with increasing activity, can be used to overcome such challenges. Initially focused on robots and now mostly applied to ground vehicles, artificial intelligence begins to be used also to train drones. Reinforcement learning is the branch of artificial intelligence able to train machines. The application of reinforcement learning to drones will provide them with more intelligence, eventually converting drones in fully-autonomous machines. In this work, reinforcement learning is studied for drone delivery. As sensors, the drone only has a stereo-vision front camera, from which depth information is obtained. The drone is trained to fly to a destination in a neighborhood environment that has plenty of obstacles such as trees, cables, cars and houses. The flying area is also delimited by a geo-fence; this is a virtual (non-visible) fence that prevents the drone from entering or leaving a defined area. The drone has to avoid visible obstacles and has to reach a goal. Results show that, in comparison with the previous results, the new algorithms have better results, not only with a better reward, but also with a reduction of its variance. The second contribution is the checkpoints. They consist of saving a trained model every time a better reward is achieved. Results show how checkpoints improve the test results.
引用
收藏
页码:1 / 19
页数:19
相关论文
共 50 条
  • [1] Deep Reinforcement Learning for Truck-Drone Delivery Problem
    Bi, Zhiliang
    Guo, Xiwang
    Wang, Jiacun
    Qin, Shujin
    Liu, Guanjun
    DRONES, 2023, 7 (07)
  • [2] Drone Deep Reinforcement Learning: A Review
    Azar, Ahmad Taher
    Koubaa, Anis
    Ali Mohamed, Nada
    Ibrahim, Habiba A.
    Ibrahim, Zahra Fathy
    Kazim, Muhammad
    Ammar, Adel
    Benjdira, Bilel
    Khamis, Alaa M.
    Hameed, Ibrahim A.
    Casalino, Gabriella
    ELECTRONICS, 2021, 10 (09)
  • [3] The Use of Deep Reinforcement Learning for Flying a Drone
    Domitran, Sandro
    Babac, Marina Bagic
    JOURNAL OF INFORMATION SCIENCE AND ENGINEERING, 2021, 37 (05) : 1165 - 1176
  • [4] Autonomous Drone Racing with Deep Reinforcement Learning
    Song, Yunlong
    Steinweg, Mats
    Kaufmann, Elia
    Scaramuzza, Davide
    2021 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2021, : 1205 - 1212
  • [5] Autonomous drone interception with Deep Reinforcement Learning
    Bertoin, David
    Gauffriau, Adrien
    Grasset, Damien
    Gupta, Jayant Sen
    CEUR Workshop Proceedings, 2022, 3173
  • [6] Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery
    Tarhan, Farabi Ahmed
    Ure, Nazim Kemal
    DRONES, 2024, 8 (03)
  • [7] Reinforcement Learning Based Truck-and-Drone Coordinated Delivery
    Wu G.
    Fan M.
    Shi J.
    Feng Y.
    IEEE Transactions on Artificial Intelligence, 2023, 4 (04): : 754 - 763
  • [8] Deep reinforcement learning for drone navigation using sensor data
    Hodge, Victoria J.
    Hawkins, Richard
    Alexander, Rob
    NEURAL COMPUTING & APPLICATIONS, 2021, 33 (06): : 2015 - 2033
  • [9] Prioritized Environment Configuration for Drone Control with Deep Reinforcement Learning
    Jang, Sooyoung
    Choi, Changbeom
    HUMAN-CENTRIC COMPUTING AND INFORMATION SCIENCES, 2022, 12
  • [10] Drone Navigation and Avoidance of Obstacles Through Deep Reinforcement Learning
    Cetin, Ender
    Barrado, Cristina
    Munoz, Guillem
    Macias, Miguel
    Pastor, Enric
    2019 IEEE/AIAA 38TH DIGITAL AVIONICS SYSTEMS CONFERENCE (DASC), 2019,