A Novel Deep Q-learning Method for Dynamic Spectrum Access

被引:4
作者
Tomovic, S. [1 ]
Radusinovic, I [1 ]
机构
[1] Univ Montenegro, Fac Elect Engn, Dzordza Vasingtona Bb, Podgorica 81000, Montenegro
来源
2020 28TH TELECOMMUNICATIONS FORUM (TELFOR) | 2020年
关键词
Cognitive radio; Reinforcement learning; OPTIMALITY;
D O I
10.1109/telfor51502.2020.9306591
中图分类号
TN [电子技术、通信技术];
学科分类号
0809 ;
摘要
In this paper, we propose a new Dynamic Spectrum Access (DSA) method for multi-channel wireless networks. We assume that DSA nodes, as secondary users, do not have prior knowledge of the system dynamics. Since DSA nodes have only partial observability of the channel states, the problem is formulated as a Partially Observable Markov Decision Process (POMDP) with exponential time complexity. We have developed a novel Deep Reinforcement Learning (DRL) based DSA method which combines a double deep Q-learning architecture with a recurrent neural network and takes advantage of a prioritized experience buffer. The simulation analysis shows that the proposed method accurately predicts the channels state based on the fixed-length history of partial observations. Compared with other DRL methods, the proposed solution is able to find a near-optimal policy in a smaller number of iterations and suits a wide range of communication environment conditions. The performance improvement increases with the number of channels and a channel state transition uncertainty.
引用
收藏
页码:9 / 12
页数:4
相关论文
共 50 条
[1]   Deep Q-Learning with Multiband Sensing for Dynamic Spectrum Access [J].
Nguyen, Ha Q. ;
Nguyen, Binh T. ;
Dong, Trung Q. ;
Ngo, Dat T. ;
Nguyen, Tuan A. .
2018 IEEE INTERNATIONAL SYMPOSIUM ON DYNAMIC SPECTRUM ACCESS NETWORKS (DYSPAN), 2018,
[2]   Deep Q-learning multiple networks based dynamic spectrum access with energy harvesting for green cognitive radio network [J].
Peng, Bao ;
Yao, Zhi ;
Liu, Xin ;
Zhou, Guofu .
COMPUTER NETWORKS, 2023, 224
[3]   Distributed dynamic spectrum access through multi-agent deep recurrent Q-learning in cognitive radio network [J].
Giri, Manish Kumar ;
Majumder, Saikat .
PHYSICAL COMMUNICATION, 2023, 58
[4]   Double Deep Q-Learning based Backhaul Spectrum Allocation in Integrated Access and Backhaul Network [J].
Park, Jeonghun ;
Jin, Heetae ;
Joo, Jaehan ;
Choi, Geonho ;
Kim, Suk Chan .
2023 INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE IN INFORMATION AND COMMUNICATION, ICAIIC, 2023, :706-708
[5]   A deep Q-learning method for optimizing visual search strategies in backgrounds of dynamic noise [J].
Zhou, Weimin ;
Eckstein, Miguel P. .
MEDICAL IMAGING 2022: IMAGE PERCEPTION, OBSERVER PERFORMANCE, AND TECHNOLOGY ASSESSMENT, 2022, 12035
[6]   Deep Coalitional Q-Learning for Dynamic Coalition Formation in Edge Computing [J].
Ding, Shiyao ;
Lin, Donghui .
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2022, E105D (05) :864-872
[7]   Distributed Deep Reinforcement Learning with Wideband Sensing for Dynamic Spectrum Access [J].
Kaytaz, Umuralp ;
Ucar, Seyhan ;
Akgun, Bans ;
Coleri, Sinem .
2020 IEEE WIRELESS COMMUNICATIONS AND NETWORKING CONFERENCE (WCNC), 2020,
[8]   A Deep Reinforcement Learning Approach to Fair Distributed Dynamic Spectrum Access [J].
Jalil, Syed Qaisar ;
Rehmani, Mubashir Husain ;
Chalup, Stephan .
PROCEEDINGS OF THE 17TH EAI INTERNATIONAL CONFERENCE ON MOBILE AND UBIQUITOUS SYSTEMS: COMPUTING, NETWORKING AND SERVICES (MOBIQUITOUS 2020), 2021, :236-244
[9]   Active deep Q-learning with demonstration [J].
Si-An Chen ;
Voot Tangkaratt ;
Hsuan-Tien Lin ;
Masashi Sugiyama .
Machine Learning, 2020, 109 :1699-1725
[10]   Active deep Q-learning with demonstration [J].
Chen, Si-An ;
Tangkaratt, Voot ;
Lin, Hsuan-Tien ;
Sugiyama, Masashi .
MACHINE LEARNING, 2020, 109 (9-10) :1699-1725