Two Tier Slicing Resource Allocation Algorithm Based on Deep Reinforcement Learning and Joint Bidding in Wireless Access Networks

被引:9
作者
Chen, Geng [1 ]
Zhang, Xu [1 ]
Shen, Fei [2 ]
Zeng, Qingtian [1 ]
机构
[1] Shandong Univ Sci & Technol, Coll Elect & Informat Engn, Qingdao 266590, Peoples R China
[2] Chinese Acad Sci, Shanghai Inst Microsyst & Informat Technol, Shanghai 200050, Peoples R China
基金
国家重点研发计划; 中国国家自然科学基金;
关键词
network slicing (NS); resource allocation; deep reinforcement learning; bidding; USER ASSOCIATION; 5G; MANAGEMENT;
D O I
10.3390/s22093495
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Network slicing (NS) is an emerging technology in recent years, which enables network operators to slice network resources (e.g., bandwidth, power, spectrum, etc.) in different types of slices, so that it can adapt to different application scenarios of 5 g network: enhanced mobile broadband (eMBB), massive machine-type communications (mMTC) and ultra-reliable and low-latency communications (URLLC). In order to allocate these sliced network resources more effectively to users with different needs, it is important that manage the allocation of network resources. Actually, in the practical network resource allocation problem, the resources of the base station (BS) are limited and the demand of each user for mobile services is different. To better deal with the resource allocation problem, more effective methods and algorithms have emerged in recent years, such as the bidding method, deep learning (DL) algorithm, ant colony algorithm (AG), and wolf colony algorithm (WPA). This paper proposes a two tier slicing resource allocation algorithm based on Deep Reinforcement Learning (DRL) and joint bidding in wireless access networks. The wireless virtual technology divides mobile operators into infrastructure providers (InPs) and mobile virtual network operators (MVNOs). This paper considers a single base station, multi-user shared aggregated bandwidth radio access network scenario and joins the MVNOs to fully utilize base station resources, and divides the resource allocation process into two tiers. The algorithm proposed in this paper takes into account both the utilization of base station (BS) resources and the service demand of mobile users (MUs). In the upper tier, each MVNO is treated as an agent and uses a combination of bidding and Deep Q network (DQN) allows the MVNO to get more resources from the base station. In the lower tier allocation process, each MVNO distributes the received resources to the users who are connected to it, which also uses the Dueling DQN method for iterative learning to find the optimal solution to the problem. The results show that in the upper tier, the total system utility function and revenue obtained by the proposed algorithm are about 5.4% higher than double DQN and about 2.6% higher than Dueling DQN; In the lower tier, the user service quality obtained by using the proposed algorithm is more stable, the system utility function and Se are about 0.5-2.7% higher than DQN and Double DQN, but the convergence is faster.
引用
收藏
页数:22
相关论文
共 38 条
[1]   Network Slicing and Softwarization: A Survey on Principles, Enabling Technologies, and Solutions [J].
Afolabi, Ibrahim ;
Taleb, Tarik ;
Samdanis, Konstantinos ;
Ksentini, Adlen ;
Flinck, Hannu .
IEEE COMMUNICATIONS SURVEYS AND TUTORIALS, 2018, 20 (03) :2429-2453
[2]   DeepEdge: A New QoE-Based Resource Allocation Framework Using Deep Reinforcement Learning for Future Heterogeneous Edge-IoT Applications [J].
AlQerm, Ismail ;
Pan, Jianli .
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, 2021, 18 (04) :3942-3954
[3]   Distributed User Association and Femtocell Allocation in Heterogeneous Wireless Networks [J].
Bayat, Siavash ;
Louie, Raymond H. Y. ;
Han, Zhu ;
Vucetic, Branka ;
Li, Yonghui .
IEEE TRANSACTIONS ON COMMUNICATIONS, 2014, 62 (08) :3027-3043
[4]   Slice as an Evolutionary Service: Genetic Optimization for Inter-Slice Res me Management in 5G Networks [J].
Bin Han ;
Ji Lianghai ;
Schotten, Hans D. .
IEEE ACCESS, 2018, 6 :33137-33147
[5]  
Buyakar TVK, 2020, INT CONF COMMUN SYST, DOI [10.1109/comsnets48256.2020.9027310, 10.1109/COMSNETS48256.2020.9027310]
[6]   Toward Tailored Resource Allocation of Slices in 6G Networks With Softwarization and Virtualization [J].
Cao, Haotong ;
Du, Jianbo ;
Zhao, Haitao ;
Luo, Daniel Xiapu ;
Kumar, Neeraj ;
Yang, Longxiang ;
Yu, F. Richard .
IEEE INTERNET OF THINGS JOURNAL, 2022, 9 (09) :6623-6637
[7]   A Comprehensive Survey on the E2E 5G Network Slicing Model [J].
Chahbar, Mohammed ;
Diaz, Gladys ;
Dandoush, Abdulhalim ;
Cerin, Christophe ;
Ghoumid, Kamal .
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, 2021, 18 (01) :49-62
[8]   Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks [J].
Cui, Jingjing ;
Liu, Yuanwei ;
Nallanathan, Arumugam .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2020, 19 (02) :729-743
[9]   Low-Complexity Distributed Radio Access Network Slicing: Algorithms and Experimental Results [J].
D'Oro, Salvatore ;
Restuccia, Francesco ;
Melodia, Tommaso ;
Palazzo, Sergio .
IEEE-ACM TRANSACTIONS ON NETWORKING, 2018, 26 (06) :2815-2828
[10]   A Survey on Beyond 5G Network With the Advent of 6G: Architecture and Emerging Technologies [J].
Dogra, Anutusha ;
Jha, Rakesh Kumar ;
Jain, Shubha .
IEEE ACCESS, 2021, 9 (09) :67512-67547