Intelligence-Based Reinforcement Learning for Dynamic Resource Optimization in Edge Computing-Enabled Vehicular Networks

被引:0
作者
Wang, Yuhang [1 ]
He, Ying [1 ]
Yu, F. Richard [1 ,2 ]
Wu, Kaishun [3 ]
Chen, Shanzhi [4 ,5 ]
机构
[1] Shenzhen Univ, Coll Comp Sci & Software Engn, Shenzhen 518060, Peoples R China
[2] Carleton Univ, Sch Informat Technol, Ottawa, ON K1S 5B6, Canada
[3] Hong Kong Univ Sci & Technol GZ, Hong Kong, Peoples R China
[4] State Key Lab Wireless Mobile Commun, Beijing 100083, Peoples R China
[5] China Informat & Commun Technol Grp Co Ltd CICT, Beijing 100079, Peoples R China
基金
中国国家自然科学基金;
关键词
Reinforcement learning; Vehicle dynamics; Heuristic algorithms; Resource management; Biological system modeling; Adaptation models; Decision making; Transportation; Inference algorithms; Dynamic scheduling; Active inference; prior knowledge; reinforcement learning; resource allocation; ACTIVE INFERENCE;
D O I
10.1109/TMC.2024.3506161
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Intelligent transportation systems demand efficient resource allocation and task offloading to ensure low-latency, high-bandwidth vehicular services. The dynamic nature of vehicular environments, characterized by high mobility and extensive interactions among vehicles, necessitates considering time-varying statistical regularities, especially in scenarios with sharp variations. Despite the widespread use of traditional reinforcement learning for resource allocation, its limitations in generalization and interpretability are evident. To overcome these challenges, we propose an Intelligence-based Reinforcement Learning (IRL) algorithm. This algorithm utilizes active inference to infer the real world and maintain an internal model by minimizing free energy. Enhancing the efficiency of active inference, we incorporate prior knowledge as macro guidance, ensuring more accurate and efficient training. By constructing an intelligence-based model, we eliminate the need for designing reward functions, aligning better with human thinking, and providing a method to reflect the learning, information transmission and intelligence accumulation processes. This approach also allows for quantifying intelligence to a certain extent. Considering the dynamic and uncertain nature of vehicular scenarios, we apply the IRL algorithm to environments with constantly changing parameters. Extensive simulations confirm the effectiveness of IRL, significantly improving the generalization and interpretability of intelligent models in vehicular networks.
引用
收藏
页码:2394 / 2406
页数:13
相关论文
共 40 条
[1]   Reinforcement Learning Interpretation Methods: A Survey [J].
Alharin, Alnour ;
Doan, Thanh-Nam ;
Sartipi, Mina .
IEEE ACCESS, 2020, 8 :171058-171077
[2]   A survey of inverse reinforcement learning: Challenges, methods and progress [J].
Arora, Saurabh ;
Doshi, Prashant .
ARTIFICIAL INTELLIGENCE, 2021, 297 (297)
[3]   Active inference, attention, and motor preparation [J].
Brown, Harriet ;
Friston, Karl J. ;
Bestmann, Sven .
FRONTIERS IN PSYCHOLOGY, 2011, 2
[4]   Joint Resource Management and Model Compression for Wireless Federated Learning [J].
Chen, Mingzhe ;
Shlezinger, Nir ;
Poor, H. Vincent ;
Eldar, Yonina C. ;
Cui, Shuguang .
IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS (ICC 2021), 2021,
[5]   Active inference and learning [J].
Friston, Karl ;
FitzGerald, Thomas ;
Rigoli, Francesco ;
Schwartenbeck, Philipp ;
O'Doherty, John ;
Pezzulo, Giovanni .
NEUROSCIENCE AND BIOBEHAVIORAL REVIEWS, 2016, 68 :862-879
[6]   Active inference and free energy [J].
Friston, Karl J. .
BEHAVIORAL AND BRAIN SCIENCES, 2013, 36 (03) :212-213
[7]  
Gallistel CR, 1999, J COGNITIVE NEUROSCI, V11, P126
[8]   Inductive biases for deep learning of higher-level cognition [J].
Goyal, Anirudh ;
Bengio, Yoshua .
PROCEEDINGS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES, 2022, 478 (2266)
[9]  
Gray RM, 2011, ENTROPY AND INFORMATION THEORY , SECOND EDITION, P395, DOI 10.1007/978-1-4419-7970-4
[10]  
Haarnoja T, 2018, PR MACH LEARN RES, V80