Intelligent edge content caching: A deep recurrent reinforcement learning method

被引:3
作者
Xu, Haitao [1 ]
Sun, Yuejun [1 ]
Gao, Jingnan [1 ]
Guo, Jianbo [2 ]
机构
[1] Hangzhou Dianzi Univ, Hangzhou, Zhejiang, Peoples R China
[2] Hangzhou Hexing Elect Co Ltd, Hangzhou, Peoples R China
基金
美国国家科学基金会;
关键词
Edge computing; Edge caching; Edge intelligence; Deep recurrent reinforcement learning; MOBILE; COMMUNICATION; COMPUTATION; FRAMEWORK;
D O I
10.1007/s12083-022-01369-6
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
With the rise of 5G network and the rapid growth of user equipment, there exists a gap between the stringent requirements of emerging applications and the actual functionality of the Internet. In particular, transmitting data over long network links imposes high costs, which can be addressed by the edge caching (EC) method. EC caches the content at the edge server to avoid the extraordinary cost of backhaul link communication. However, in existing EC efforts, it is common to assume either known content popularity or a two-phase caching that is predicted content popularity prior to the caching action, the former being less feasible and the latter increasing the cost of deployment to the real world. A caching strategy is proposed in this paper to cope with this problem that can be feasible end-to-end deployed and has a lower caching cost. Specifically, we first investigate the system cost, including network communication cost, cache over storage cost, and cache replacement cost. And we model the EC problem as a Markov Decision Process (MDP). Then, the Double Deep Recurrent Q Network (DDRQN) algorithm is studied to solve the EC-based MDP problem. Finally, compared with other intelligent caching strategies, the proposed caching strategy can improve the system reward by up to 24% and the cache hit rate by up to 22% under certain conditions.
引用
收藏
页码:2619 / 2632
页数:14
相关论文
共 50 条
[31]   Federated Multi-Agent Reinforcement Learning for Collaborative Edge Caching in Content Delivery Networks [J].
Chang, Jialing ;
Zhang, Naifu ;
Tao, Meixia ;
Tuo, Hu .
2022 14TH INTERNATIONAL CONFERENCE ON WIRELESS COMMUNICATIONS AND SIGNAL PROCESSING, WCSP, 2022, :166-170
[32]   Dynamic and intelligent edge server placement based on deep reinforcement learning in mobile edge computing [J].
Jiang, Xiaohan ;
Hou, Peng ;
Zhu, Hongbin ;
Li, Bo ;
Wang, Zongshan ;
Ding, Hongwei .
AD HOC NETWORKS, 2023, 145
[33]   Deep Learning-based Predictive Caching in the Edge of a Network [J].
Rahman, Saidur ;
Alam, Md. Golam Rabiul ;
Rahman, Md. Mahbubur .
2020 34TH INTERNATIONAL CONFERENCE ON INFORMATION NETWORKING (ICOIN 2020), 2020, :797-801
[34]   Joint Content Caching and Recommendation in Opportunistic Mobile Networks Through Deep Reinforcement Learning and Broad Learning [J].
Yu, Dongjin ;
Wu, Tong ;
Liu, Chengfei ;
Wang, Dongjing .
IEEE TRANSACTIONS ON SERVICES COMPUTING, 2023, 16 (04) :2727-2741
[35]   Multi-Agent Deep Reinforcement Learning for Cooperative Edge Caching via Hybrid Communication [J].
Wang, Fei ;
Emara, Salma ;
Kaplan, Isidor ;
Li, Baochun ;
Zeyl, Timothy .
ICC 2023-IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS, 2023, :1206-1211
[36]   DIMA: Distributed cooperative microservice caching for internet of things in edge computing by deep reinforcement learning [J].
Tian, Hao ;
Xu, Xiaolong ;
Lin, Tingyu ;
Cheng, Yong ;
Qian, Cheng ;
Ren, Lei ;
Bilal, Muhammad .
WORLD WIDE WEB-INTERNET AND WEB INFORMATION SYSTEMS, 2022, 25 (05) :1769-1792
[37]   A Transferable Edge Caching Method Based On Reinforcement Learning for Dense Small Cell Network [J].
Hu, Liyun ;
Fan, Shaoshuai ;
Tian, Hui .
2020 IEEE 31ST ANNUAL INTERNATIONAL SYMPOSIUM ON PERSONAL, INDOOR AND MOBILE RADIO COMMUNICATIONS (IEEE PIMRC), 2020,
[38]   Deep Reinforcement Learning-Based Edge Caching in Single-Cell Wireless Networks [J].
Wu, Rong ;
Li, Qiang ;
Ge, Xiaohu .
2020 12TH INTERNATIONAL CONFERENCE ON WIRELESS COMMUNICATIONS AND SIGNAL PROCESSING (WCSP), 2020, :298-303
[39]   Deep Multi-Agent Reinforcement Learning Based Cooperative Edge Caching in Wireless Networks [J].
Zhong, Chen ;
Gursoy, M. Cenk ;
Velipasalar, Senem .
ICC 2019 - 2019 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS (ICC), 2019,
[40]   A Graph-based Spatial-Temporal Deep Reinforcement Learning Model for Edge Caching [J].
Hou, Jiacheng ;
Nayak, Amiya .
IEEE CONFERENCE ON GLOBAL COMMUNICATIONS, GLOBECOM, 2023, :6456-6461