Scalable AP Clustering With Deep Reinforcement Learning for Cell-Free Massive MIMO

Cell-free massive MIMO (CF-mMIMO) is a promising approach for future mobile networks, utilizing centralized MIMO processing for densely distributed access points (APs). In CF-mMIMO, to reduce the computational load for signal processing while meeting throughput demands, user equipment (UEs) are served by a selected number of APs. A significant challenge is AP clustering for each UE, particularly in dynamic environments with moving UEs. One approach for optimizing the AP cluster involves deep reinforcement learning (DRL). However, with numerous UEs and APs, the computational load of DRL increases due to the larger model size and higher inference frequency. To address this, we propose an AP clustering method using distributed DRL. The model focuses on determining the AP cluster for every single UE to prevent model size expansion. The per-user models act as distributed actors, enabling parallel inference. Furthermore, to suppress inference frequency, multiple UEs with low mobility are assigned to the same actor, minimizing the number of parallel actors required without compromising throughput. Numerical simulation shows that our proposed method achieves efficient AP clustering that satisfies throughput requirements with reduced computational load in DRL, even in large-scale environments.