Distributed Q-Learning for Interference Mitigation in Self-Organised Femtocell Networks: Synchronous or Asynchronous?

Femtocells promise to improve the quality of indoor wireless communications substantially. However, a serious interference problem arises with universal frequency reuse. In this paper, an asynchronous dynamic power allocation among femtocells based on Q-learning is proposed to mitigate the interference in the network. Simulation results show that in the high femtocells density deployment, asynchronous decision-making process has better performance than the synchronous one in terms of both performance degradation of the macrocell and average capacity of femtocells. In addition, it is shown that our method has superiority to smart power control algorithm proposed by 3GPP when femtocell occupation ratio is over 53 %.