Intelligent anti-jamming decision algorithm based on proximal policy optimization

被引：0
作者：
Ma, Song [1 ,2 ]
Li, Li [3 ]
Li, Wei [2 ]
Huang, Wei [2 ]
Wang, Jun [2 ]
机构：
[1] Southwest China Institute of Electronic Technology, Chengdu
[2] National Key Laboratory of Wireless Communications, University of Electronic Science and Technology of China, Chengdu
[3] Southwest China Research Institute of Electronic Equipment, Chengdu
来源：
Tongxin Xuebao/Journal on Communications | 2024年 / 45卷 / 08期
基金：
中国国家自然科学基金;
关键词：
anti-jamming decision; deep reinforcement learning; intelligent anti-jamming; proximal policy optimization;
D O I：
10.11959/j.issn.1000-436x.2024137
中图分类号：
学科分类号：
摘要：
The existing intelligent anti-jamming methods based on deep reinforcement learning are applied to space-ground TT&C and communication links, in which the deep neural network used for decision-making has a complex structure, and the resources of satellites and other vehicles are limited, making it difficult to independently complete the timely training of complex neural network under the constraints of limited complexity, and the decision-making of anti-jamming cannot converge. Aiming at the above problems, an intelligent anti-jamming decision algorithm based on proximal policy optimization was proposed, which deployed the decision-making neural network and the training neural network in the vehicles and the ground station, respectively. The ground station conducted the optimal offline training based on the empirical information feedback from the vehicles, and assisted the decision-making neural network in parameter updating, thereby achieving the effective selection of anti-jamming strategies while satisfying the resource constraints of the vehicles. The simulation results demonstrate that the convergence speed of the proposed algorithm is increased by 37%, and the system capacity after convergence is increased by 25%, compared with the decision algorithms of policy gradient and deep Q-learning. © 2024 Editorial Board of Journal on Communications. All rights reserved.
引用
收藏
页码：249 / 257
页数：8
相关论文
共 24 条
[11]  
LI S Q, CHENG Y F, DONG B H, Et al., Research on intelligent anti-jam communication techniques, Radio Communications Technology, 38, 1, pp. 1-4, (2012)
[12]  
ZHANG M J, ZHAO R, WANG P C, Et al., Anti-jamming algorithm with reinforcement learning in UAV-aided Internet of things, Journal of Signal Processing, 37, 1, pp. 11-18, (2021)
[13]  
WANG R D, ZHANG Y L, WEI P, Et al., Intelligent anti-jamming strategy for tactical frequency-hopping system, Journal of Signal Processing, 39, 1, pp. 84-95, (2023)
[14]  
HAN G A, XIAO L, POOR H V., Two-dimensional anti-jamming communication based on deep reinforcement learning, Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2087-2091, (2017)
[15]  
LIU X, XU Y H, JIA L L, Et al., Anti-jamming communications using spectrum waterfall: a deep reinforcement learning approach, IEEE Communications Letters, 22, 5, pp. 998-1001, (2018)
[16]  
LI W, WANG J, LI L, Et al., Intelligent anti-jamming communication with continuous action decision for ultra-dense network, Proceedings of the ICC 2019-2019 IEEE International Conference on Communications (ICC), pp. 1-7, (2019)
[17]  
ZHANG M Y, DOU Y J, CHEN Z Y, Et al., Review of deep reinforcement learning and its applications in military field, Systems Engineering and Electronics, 46, 4, pp. 1297-1308, (2024)
[18]  
TANG S Q, PAN Z S, HU G Y, Et al., Application of deep reinforcement learning in space information network—status quo and prospects, Systems Engineering and Electronics, 45, 3, pp. 886-901, (2023)
[19]  
STRASSER M, POPPER C, CAPKUN S., Efficient uncoordinated FHSS anti-jamming communication, Proceedings of the Tenth ACM International Symposium on Mobile Ad Hoc Networking and Computing, pp. 207-218, (2009)
[20]  
WILHELM M, MARTINOVIC I, SCHMITT J B, Et al., Short paper: reactive jamming in wireless networks: how realistic is the threat?, Proceedings of the Fourth ACM Conference on Wireless Network Security, pp. 47-52, (2011)
123