Q-learning Based Co-operative Spectrum Mobility in Cognitive Radio Networks

In cognitive radio systems, fast and efficient spectrum selection is a vital task to minimize the overhead of spectrum scanning, and hence to improve the response time of the system. So, the choice of channel sensing sequence plays an important role for better performance of the system. This paper proposes a co-operative Q-learning based spectrum sensing technique for the secondary users of an ad hoc network to access the primary channels. By the proposed technique, every secondary user (SU) maintains a dynamic priority list of channels based on Q-learning from its own action-observation history, as well as from spatial channel information exchange among its local neighbors. Whenever there is a demand an SU scans the spectrum according to the order in the priority list until there is a success. Simulation studies show that with significantly less computing and scanning overhead, our proposed Q-learning based approach improves the response time and call block / drop rate to offer better performance compared to other contemporary reinforcement learning based approaches.