DGTRL: Deep graph transfer reinforcement learning method based on fusion of knowledge and data

被引:2
作者
Chen, Genxin [1 ]
Qi, Jin [2 ]
Gao, Yu [1 ]
Zhu, Xingjian [1 ]
Dong, Zhenjiang [3 ]
Sun, Yanfei [2 ]
机构
[1] Nanjing Univ Posts & Telecommun, Sch Automat, Nanjing, Jiangsu, Peoples R China
[2] Nanjing Univ Posts & Telecommun, Sch Internet Things, Nanjing, Jiangsu, Peoples R China
[3] Nanjing Univ Posts & Telecommun, Sch Comp Sci, Nanjing, Jiangsu, Peoples R China
基金
中国国家自然科学基金;
关键词
Knowledge reuse; Deep reinforcement learning; Graph matching; Transfer learning; Adaptive policy transfer; Fusion; NETWORK;
D O I
10.1016/j.ins.2023.120019
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Deep reinforcement learning has shown promising application effects in many fields. However, issues such as low sample efficiency and weak knowledge transfer and generalization capabilities constrain its in-depth development. This paper proposes a deep graph transfer reinforcement learning method based on knowledge and data fusion. Through knowledge transfer between tasks and data training in exploration, efficient learning driven by knowledge and data fusion is realized. First, we map the trajectories of agents in the source and target domains into graph structures to represent knowledge and environmental models and use a graph matching method to match the knowledge adaptation domain based on similarity. Second, real-time transfer of source domain knowledge is carried out in a similar mapping domain, and policy mapping, verification, and adaptive transfer mechanisms based on knowledge reuse frequency are designed to effectively improve the transfer effect. Finally, we evaluate and validate the proposed algorithm, mechanism, and their application effectiveness in diverse and multitask knowledge transfer scenarios using three indicators in environments such as maze. The results show that the method improves the target task time indicators by 84.7% and 70% under the conditions of no verification exploration and a standard transfer setting, respectively, achieving significant knowledge reuse effects.
引用
收藏
页数:21
相关论文
共 50 条
[1]  
Brys T, 2015, PROCEEDINGS OF THE 2015 INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS & MULTIAGENT SYSTEMS (AAMAS'15), P181
[2]  
Chemali J, 2015, PROCEEDINGS OF THE TWENTY-FOURTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE (IJCAI), P3380
[3]  
Chen Xiang, 2022, Advances in Neural Information Processing Systems
[4]  
Cho M, 2010, LECT NOTES COMPUT SC, V6315, P492
[5]  
Cho YJ, 2022, PROCEEDINGS OF THE THIRTY-FIRST INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2022, P2881
[6]  
Fatemi B, 2023, J MACH LEARN RES, V24
[7]  
Finkelstein M., 2022, Advances in Neural Information Processing Systems, V35, P34039
[8]   Information Optimization and Transferable State Abstractions in Deep Reinforcement Learning [J].
Gomez, Diego ;
Quijano, Nicanor ;
Giraldo, Luis Felipe .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2023, 45 (04) :4782-4793
[9]  
Gulcehre C., 2020, PROC INT C LEARN REP, P1
[10]   A new multi-source Transfer Learning method based on Two-stage Weighted Fusion [J].
Huang, Linqing ;
Fan, Jinfu ;
Zhao, Wangbo ;
You, Yang .
KNOWLEDGE-BASED SYSTEMS, 2023, 262