Energy-efficient UAV-enabled computation offloading for industrial internet of things: a deep reinforcement learning approach

被引:7
作者
Shi, Shuo [1 ,3 ]
Wang, Meng [1 ]
Gu, Shushi [2 ,3 ]
Zheng, Zhong [4 ]
机构
[1] Harbin Inst Technol, Sch Elect & Informat Engn, Harbin 150001, Peoples R China
[2] Harbin Inst Technol Shenzhen, Sch Elect & Informat Engn, Shenzhen 518055, Peoples R China
[3] Peng Cheng Lab, Shenzhen 518055, Peoples R China
[4] Int Innovat Inst HIT Huizhou, Huizhou 516000, Guangdong, Peoples R China
关键词
Energy efficiency; Deep reinforcement learning; Computation offloading; Mobile edge computing; Unmanned aerial vehicles; EDGE; WIRELESS;
D O I
10.1007/s11276-021-02789-7
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Industrial Internet of Things (IIoT) has been envisioned as a killer application of 5G and beyond. However, due to the shortness of computation ablility and batery capacity, it is challenging for IIoT devices to process latency-sensitive and resource-sensitive tasks. Moblie Edge Computing (MEC), as a promising paradigm for handling tasks with high quality of service (QoS) requirement and for energy-constrained IIoT devices, allows IIoT devices to offload their tasks to MEC servers, which can significantly reduce the task process delay and energy consumptions. However, the deployment of the MEC servers rely heavily on communication infrastructure, which greatly reduce the flexibility. Toward this end, in this paper, we consider multiple Unmanned Aerial Vehicles (UAV) eqqipped with transceivers as aerial MEC servers to provide IIoT devices computation offloading opportunities due to their high controbility. IIoT devices can choose to offload the tasks to UAVs through air-ground links, or to offload the tasks to the remote cloud center through ground cellular network, or to process the tasks locally. We formulate the multi-UAV-Enabled computation offloading problem as a mixed integer non-linear programming (MINLP) problem and prove its NP-hardness. To obtain the energy-efficient and low complexity solution, we propose an intelligent computation offloading algorithm called multi-agent deep Q-learning with stochastic prioritized replay (MDSPR). Numerical results show that the proposed MDSPR converges fast and outperforms the benchmark algorithms, including random method, deep Q-learning method and double deep Q-learning method in terms of energy efficiency and task successful rate.
引用
收藏
页码:3921 / 3934
页数:14
相关论文
共 22 条
[1]   Optimal LAP Altitude for Maximum Coverage [J].
Al-Hourani, Akram ;
Kandeepan, Sithamparanathan ;
Lardner, Simon .
IEEE WIRELESS COMMUNICATIONS LETTERS, 2014, 3 (06) :569-572
[2]   Optimized Computation Offloading Performance in Virtual Edge Computing Systems via Deep Reinforcement Learning [J].
Chen, Xianfu ;
Zhang, Honggang ;
Wu, Celimuge ;
Mao, Shiwen ;
Ji, Yusheng ;
Bennis, Mehdi .
IEEE INTERNET OF THINGS JOURNAL, 2019, 6 (03) :4005-4018
[3]   Collaborative Computation Offloading for Multiaccess Edge Computing Over Fiber-Wireless Networks [J].
Guo, Hongzhi ;
Liu, Jiajia .
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2018, 67 (05) :4514-4526
[4]   UAV-Assisted Relaying and Edge Computing: Scheduling and Trajectory Optimization [J].
Hu, Xiaoyan ;
Wong, Kai-Kit ;
Yang, Kun ;
Zheng, Zhongbin .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2019, 18 (10) :4738-4752
[5]   Deep Reinforcement Learning for Offloading and Resource Allocation in Vehicle Edge Computing and Networks [J].
Liu, Yi ;
Yu, Huimin ;
Xie, Shengli ;
Zhang, Yan .
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2019, 68 (11) :11158-11168
[6]   Edge QoE: Computation Offloading With Deep Reinforcement Learning for Internet of Things [J].
Lu, Haodong ;
He, Xiaoming ;
Du, Miao ;
Ruan, Xiukai ;
Sun, Yanfei ;
Wang, Kun .
IEEE INTERNET OF THINGS JOURNAL, 2020, 7 (10) :9255-9265
[7]   Applications of Deep Reinforcement Learning in Communications and Networking: A Survey [J].
Luong, Nguyen Cong ;
Hoang, Dinh Thai ;
Gong, Shimin ;
Niyato, Dusit ;
Wang, Ping ;
Liang, Ying-Chang ;
Kim, Dong In .
IEEE COMMUNICATIONS SURVEYS AND TUTORIALS, 2019, 21 (04) :3133-3174
[8]  
Mao YJ, 2017, INT GEOL REV, V59, P1276, DOI [10.1080/00206814.2016.1209435, 10.1109/COMST.2017.2745201]
[9]   Noncooperative Cellular Wireless with Unlimited Numbers of Base Station Antennas [J].
Marzetta, Thomas L. .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2010, 9 (11) :3590-3600
[10]   Human-level control through deep reinforcement learning [J].
Mnih, Volodymyr ;
Kavukcuoglu, Koray ;
Silver, David ;
Rusu, Andrei A. ;
Veness, Joel ;
Bellemare, Marc G. ;
Graves, Alex ;
Riedmiller, Martin ;
Fidjeland, Andreas K. ;
Ostrovski, Georg ;
Petersen, Stig ;
Beattie, Charles ;
Sadik, Amir ;
Antonoglou, Ioannis ;
King, Helen ;
Kumaran, Dharshan ;
Wierstra, Daan ;
Legg, Shane ;
Hassabis, Demis .
NATURE, 2015, 518 (7540) :529-533