Energy Efficiency Optimization in Heterogeneous Networks Based on Deep Reinforcement Learning

To meet the rapid increasing requirement of service and application of communication, heterogeneous wireless networks with the macrocell and the femtocells are considered in this paper. How to deal with resource allocation efficiently and dynamically in heterogeneous wireless networks puts forward to an urgent problem. In this paper, we aim to maximize the overall network's energy efficiency, where multiple femto base stations are randomly and densely distributed in the coverage of macro base station. We first construct an EE model and formulate the problem for optimization. Then we propose a Nature DQN algorithm in deep reinforcement learning to solve it with power discretization. Finally, in simulations, it demonstrates that the proposed Nature DQN can not only achieve better energy efficiency than Q-learning and water-filling but also accelerate the convergence.