A Deep Reinforcement Learning-Based Resource Management Game in Vehicular Edge Computing

被引:67
作者
Zhu, Xiaoyu [1 ]
Luo, Yueyi [2 ,3 ]
Liu, Anfeng [1 ]
Xiong, Neal N. [4 ]
Dong, Mianxiong [5 ]
Zhang, Shaobo [6 ]
机构
[1] Cent South Univ, Sch Comp Sci & Engn, Changsha 410083, Peoples R China
[2] Cent South Univ, Sch Math & Stat, Changsha 410083, Peoples R China
[3] Cent South Univ, Network Resources Management & Trust Evaluat, Key Lab Hunan Prov, Changsha 410083, Peoples R China
[4] Northeastern State Univ, Dept Math & Comp Sci, Tahlequah, OK 74464 USA
[5] Muroran Inst Technol, Dept Sci & Informat, Muroran, Hokkaido 0508585, Japan
[6] Hunan Univ Sci & Technol, Sch Comp Sci & Engn, Xiangtan 411201, Peoples R China
基金
中国国家自然科学基金;
关键词
Resource management; Servers; Games; Edge computing; Reinforcement learning; Pricing; Computational modeling; Stackelberg pricing game; deep reinforcement learning; vehicular edge computing; INCENTIVE MECHANISMS; ALLOCATION; NETWORKS; RADIO;
D O I
10.1109/TITS.2021.3114295
中图分类号
TU [建筑科学];
学科分类号
0813 ;
摘要
Vehicular Edge Computing (VEC) is a promising paradigm that leverages the vehicles to offload computation tasks to the nearby VEC server with the aim of supporting the low latency vehicular application scenarios. Incentivizing VEC servers to participate in computation offloading activities and make full use of computation resources is of great importance to the success of intelligent transportation services. In this paper, we formulate the competitive interactions between the VEC servers and vehicles as a two-stage Stackelberg game with the VEC servers as the leader players and the vehicles as the followers. After obtaining the full information of vehicles, the VEC server calculates the unit price of computation resource. Given the unit prices announced by VEC server, the vehicles determine the amount of computation resource to purchase from VEC server. In the scenario that vehicles do not want to share their computation demands, a deep reinforcement learning based resource management scheme is proposed to maximize the profits of vehicles and VEC server. The extensive experimental results have demonstrated the effectiveness of our proposed resource management scheme based on Stackelberg game and deep reinforcement learning.
引用
收藏
页码:2422 / 2433
页数:12
相关论文
共 45 条
[1]   Toward Pre-Empted EV Charging Recommendation Through V2V-Based Reservation System [J].
Cao, Yue ;
Jiang, Tao ;
Kaiwartya, Omprakash ;
Sun, Hongjian ;
Zhou, Huan ;
Wang, Ran .
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2021, 51 (05) :3026-3039
[2]   Intelligent resource allocation management for vehicles network: An A3C learning approach [J].
Chen, Miaojiang ;
Wang, Tian ;
Ota, Kaoru ;
Dong, Mianxiong ;
Zhao, Ming ;
Liu, Anfeng .
COMPUTER COMMUNICATIONS, 2020, 151 :485-494
[3]   Joint Load Balancing and Offloading in Vehicular Edge Computing and Networks [J].
Dai, Yueyue ;
Xu, Du ;
Maharjan, Sabita ;
Zhang, Yan .
IEEE INTERNET OF THINGS JOURNAL, 2019, 6 (03) :4377-4387
[4]   Computation Offloading and Resource Allocation in Vehicular Networks Based on Dual-Side Cost Minimization [J].
Du, Jianbo ;
Yu, F. Richard ;
Chu, Xiaoli ;
Feng, Jie ;
Lu, Guangyue .
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2019, 68 (02) :1079-1092
[5]   An actor-critic reinforcement learning-based resource management in mobile edge computing systems [J].
Fu, Fang ;
Zhang, Zhicai ;
Yu, Fei Richard ;
Yan, Qiao .
INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2020, 11 (08) :1875-1889
[6]   Joint Radio and Computational Resource Allocation in IoT Fog Computing [J].
Gu, Yunan ;
Chang, Zheng ;
Pan, Miao ;
Song, Lingyang ;
Han, Zhu .
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2018, 67 (08) :7475-7484
[7]   FiWi ENHANCED VEHICULAR EDGE COMPUTING NETWORKS Collaborative Computation Task Offloading [J].
Guo, Hongzhi ;
Zhang, Jie ;
Liu, Jiajia .
IEEE VEHICULAR TECHNOLOGY MAGAZINE, 2019, 14 (01) :45-53
[8]   Software-Defined Networks with Mobile Edge Computing and Caching for Smart Cities: A Big Data Deep Reinforcement Learning Approach [J].
He, Ying ;
Yu, F. Richard ;
Zhao, Nan ;
Leung, Victor C. M. ;
Yin, Hongxi .
IEEE COMMUNICATIONS MAGAZINE, 2017, 55 (12) :31-37
[9]  
Josilo S, 2019, IEEE INFOCOM SER, P2467, DOI [10.1109/infocom.2019.8737480, 10.1109/INFOCOM.2019.8737480]
[10]   Edge Cloud-Enabled Radio Resource Management for Co-Operative Automated Driving [J].
Keshavamurthy, Prajwal ;
Pateromichelakis, Emmanouil ;
Dahlhaus, Dirk ;
Zhou, Chan .
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 2020, 38 (07) :1515-1530