A Novel Deep Q-learning Method for Dynamic Spectrum Access

In this paper, we propose a new Dynamic Spectrum Access (DSA) method for multi-channel wireless networks. We assume that DSA nodes, as secondary users, do not have prior knowledge of the system dynamics. Since DSA nodes have only partial observability of the channel states, the problem is formulated as a Partially Observable Markov Decision Process (POMDP) with exponential time complexity. We have developed a novel Deep Reinforcement Learning (DRL) based DSA method which combines a double deep Q-learning architecture with a recurrent neural network and takes advantage of a prioritized experience buffer. The simulation analysis shows that the proposed method accurately predicts the channels state based on the fixed-length history of partial observations. Compared with other DRL methods, the proposed solution is able to find a near-optimal policy in a smaller number of iterations and suits a wide range of communication environment conditions. The performance improvement increases with the number of channels and a channel state transition uncertainty.