Multi-agent deep reinforcement learning for computation offloading in cooperative edge network

被引:4
作者
Wu, Pengju [1 ]
Guan, Yepeng [1 ,2 ,3 ]
机构
[1] Shanghai Univ, Sch Commun & Informat Engn, Shanghai 200444, Peoples R China
[2] Minist Educ, Key Lab Adv Display & Syst Applicat, Shanghai 200072, Peoples R China
[3] Shanghai Univ, Minist Educ, Key Lab Silicate Cultural Rel Conservat, Shanghai 200444, Peoples R China
基金
国家重点研发计划;
关键词
Mobile edge computing; Multi-agent deep reinforcement learning; Computation offloading; RESOURCE-ALLOCATION; MOBILE; OPTIMIZATION; CLOUD;
D O I
10.1007/s10844-024-00907-3
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Mobile Edge Computing (MEC) has emerged as an effective paradigm for reducing latency and enhancing computational efficiency. However, the rapid proliferation of edge servers and user devices has significantly increased the complexity of task processing and resource management. Traditional task offloading approaches often rely on centralized decision-making, resulting in high computational complexity and time costs. To address these challenges, this paper introduces a dynamic collaborative framework involving multiple users and edge servers. We formulate the problem of resource allocation and task offloading as a multi-objective Markov Decision Process (MDP) with a mixed action space. To solve this, we propose a novel algorithm called Multi-Agent Mobile Edge Computing (MA-MEC), which leverages multi-agent reinforcement learning. In MA-MEC, each mobile edge server (MES) operates as an independent learning agent. Through centralized training and decentralized execution, these agents collaborate to develop efficient task offloading strategies in complex and dynamic edge environments. Simulation results demonstrate the effectiveness of our approach. MES agents learn to execute tasks more efficiently, increasing the number of processed tasks by 12.5%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document}, while task offloading rates rise by 17%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document}, and time costs are reduced by 53%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document} compared to baseline methods. The proposed method shows significant advantages, especially in resource-constrained scenarios.
引用
收藏
页码:567 / 591
页数:25
相关论文
共 45 条
[1]   Mobile Edge Computing: A Survey [J].
Abbas, Nasir ;
Zhang, Yan ;
Taherkordi, Amir ;
Skeie, Tor .
IEEE INTERNET OF THINGS JOURNAL, 2018, 5 (01) :450-465
[2]   Mobile Edge Offloading Using Markov Decision Processes [J].
Alasmari, Khalid R. ;
Green, Robert C., II ;
Alam, Mansoor .
EDGE COMPUTING - EDGE 2018, 2018, 10973 :80-90
[3]   D3PG: Dirichlet DDPG for Task Partitioning and Offloading With Constrained Hybrid Action Space in Mobile-Edge Computing [J].
Ale, Laha ;
King, Scott A. ;
Zhang, Ning ;
Sattar, Abdul Rahman ;
Skandaraniyam, Janahan .
IEEE INTERNET OF THINGS JOURNAL, 2022, 9 (19) :19260-19272
[4]   Joint User Scheduling and Computing Resource Allocation Optimization in Asynchronous Mobile Edge Computing Networks [J].
Cang, Yihan ;
Chen, Ming ;
Pan, Yijin ;
Yang, Zhaohui ;
Hu, Ye ;
Sun, Haijian ;
Chen, Mingzhe .
IEEE TRANSACTIONS ON COMMUNICATIONS, 2024, 72 (06) :3378-3392
[5]   Task Offloading for Mobile Edge Computing in Software Defined Ultra-Dense Network [J].
Chen, Min ;
Hao, Yixue .
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 2018, 36 (03) :587-597
[6]   Multi-agent deep reinforcement learning for collaborative task offloading in mobile edge computing networks [J].
Chen, Minxuan ;
Guo, Aihuang ;
Song, Chunlin .
DIGITAL SIGNAL PROCESSING, 2023, 140
[7]   A Stackelberg game approach to multiple resources allocation and pricing in mobile edge computing [J].
Chen, Yifan ;
Li, Zhiyong ;
Yang, Bo ;
Nai, Ke ;
Li, Keqin .
FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2020, 108 :273-287
[8]   Traffic-Aware Lightweight Hierarchical Offloading Toward Adaptive Slicing-Enabled SAGIN [J].
Chen, Zheyi ;
Zhang, Junjie ;
Min, Geyong ;
Ning, Zhaolong ;
Li, Jie .
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 2024, 42 (12) :3536-3550
[9]   Joint Computation Offloading and Resource Allocation in Multi-Edge Smart Communities With Personalized Federated Deep Reinforcement Learning [J].
Chen, Zheyi ;
Xiong, Bing ;
Chen, Xing ;
Min, Geyong ;
Li, Jie .
IEEE TRANSACTIONS ON MOBILE COMPUTING, 2024, 23 (12) :11604-11619
[10]   Profit-Aware Cooperative Offloading in UAV-Enabled MEC Systems Using Lightweight Deep Reinforcement Learning [J].
Chen, Zheyi ;
Zhang, Junjie ;
Zheng, Xianghan ;
Min, Geyong ;
Li, Jie ;
Rong, Chunming .
IEEE INTERNET OF THINGS JOURNAL, 2024, 11 (12) :21325-21336