Energy-Efficient Sleep-Mode Based on Deep Reinforcement Learning for Cell-Free mmWave Massive MIMO Systems

To improve the global energy-efficiency (GEE) performance in cell-free millimeter-wave massive MIMO (CF mmWave mMIMO) systems, the access points (APs) sleep-mode techniques in dynamic time-varying channels are investigated. The AP switch ON-OFF (ASO) strategy is formulated as a Markov decision process. Thus, a deep reinforcement learning (DRL) model can be used to solve the AP activation problem. The interference-aware method and the locality-sensitive hashing method are introduced to reduce sample bias and interaction between agents and complex environments. A novel cost function is constructed to achieve a better balance between GEE and achievable rate under the strict quality of service (QoS) constraints. In order to accelerate the convergence of the dueling deep Q-Network (DQN), the state space is mapped to the smaller hierarchical state space by discretizing the cost function. Simulation results have demonstrated the performance advantage of the convergence of deep reinforcement learning and GEE under the strict QoS constraint. © 2023 Chinese Institute of Electronics. All rights reserved.