Path Planning for UAV-Mounted Mobile Edge Computing With Deep Reinforcement Learning

被引:230
作者
Liu, Qian [1 ]
Shi, Long [2 ]
Sun, Linlin [1 ]
Li, Jun [1 ]
Ding, Ming [3 ]
Shu, Feng [1 ]
机构
[1] Nanjing Univ Sci & Technol, Sch Elect & Opt Engn, Nanjing 210094, Peoples R China
[2] Singapore Univ Technol & Design, Sci & Math Cluster, Singapore 487372, Singapore
[3] CSIRO, Data61, Sydney, NSW 2015, Australia
关键词
Unmanned aerial vehicle; edge computing; path planning; Markov decision process; deep reinforcement learning; TRAJECTORY DESIGN; NETWORKS;
D O I
10.1109/TVT.2020.2982508
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
In this letter, we study an unmanned aerial vehicle (UAV)-mounted mobile edge computing network, where the UAV executes computational tasks offloaded from mobile terminal users (TUs) and the motion of each TU follows a Gauss-Markov random model. To ensure the quality-of-service (QoS) of each TU, the UAV with limited energy dynamically plans its trajectory according to the locations of mobile TUs. Towards this end, we formulate the problem as a Markov decision process, wherein the UAV trajectory and UAV-TU association are modeled as the parameters to be optimized. To maximize the system reward and meet the QoS constraint, we develop a QoS-based action selection policy in the proposed algorithm based on double deep Q-network. Simulations show that the proposed algorithm converges more quickly and achieves a higher sum throughput than conventional algorithms.
引用
收藏
页码:5723 / 5728
页数:6
相关论文
共 21 条
[1]   Mobile Edge Computing: A Survey [J].
Abbas, Nasir ;
Zhang, Yan ;
Taherkordi, Amir ;
Skeie, Tor .
IEEE INTERNET OF THINGS JOURNAL, 2018, 5 (01) :450-465
[2]  
[Anonymous], 2019, IEEE INTERNET THINGS, DOI DOI 10.1109/JIOT.2019.2923702
[3]   Mobility Models, Traces and Impact of Mobility on Opportunistic Routing Algorithms: A Survey [J].
Batabyal, Suvadip ;
Bhaumik, Parama .
IEEE COMMUNICATIONS SURVEYS AND TUTORIALS, 2015, 17 (03) :1679-1707
[4]   Space/Aerial-Assisted Computing Offloading for IoT Applications: A Learning-Based Approach [J].
Cheng, Nan ;
Lyu, Feng ;
Quan, Wei ;
Zhou, Conghao ;
He, Hongli ;
Shi, Weisen ;
Shen, Xuemin .
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 2019, 37 (05) :1117-1129
[5]   Fuzzy Q learning algorithm for dual-aircraft path planning to cooperatively detect targets by passive radars [J].
Gao, Xiang ;
Fang, Yangwang ;
Wu, Youli .
JOURNAL OF SYSTEMS ENGINEERING AND ELECTRONICS, 2013, 24 (05) :800-810
[6]   Mobile Edge Computing via a UAV-Mounted Cloudlet: Optimization of Bit Allocation and Path Planning [J].
Jeong, Seongah ;
Simeone, Osvaldo ;
Kang, Joonhyuk .
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2018, 67 (03) :2049-2063
[7]   Non-convex optimization and rate control for multi-class services in the Internet [J].
Lee, JW ;
Mazumdar, RR ;
Shroff, NB .
IEEE-ACM TRANSACTIONS ON NETWORKING, 2005, 13 (04) :827-840
[8]   Trajectory Design and Power Control for Multi-UAV Assisted Wireless Networks: A Machine Learning Approach [J].
Liu, Xiao ;
Liu, Yuanwei ;
Chen, Yue ;
Hanzo, Lajos .
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2019, 68 (08) :7957-7969
[9]   Mobile Unmanned Aerial Vehicles (UAVs) for Energy-Efficient Internet of Things Communications [J].
Mozaffari, Mohammad ;
Saad, Walid ;
Bennis, Mehdi ;
Debbah, Merouane .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2017, 16 (11) :7574-7589
[10]   User Association and Path Planning for UAV-Aided Mobile Edge Computing With Energy Restriction [J].
Qian, Yuwen ;
Wang, Feifei ;
Li, Jun ;
Shi, Long ;
Cai, Kui ;
Shu, Feng .
IEEE WIRELESS COMMUNICATIONS LETTERS, 2019, 8 (05) :1312-1315