Accelerated Deep Reinforcement Learning for Uplink Power Control in a Dynamic Cell-Free Massive MIMO Network

We investigate the deep reinforcement learning (DRL) framework for uplink power control in a cell-free massive multiple-input, multiple-output (MIMO) network. Although DRL does not require prior sets of training data as opposed to supervised or unsupervised machine learning approaches, existing methods suffer from substantial convergence time, which is prohibitive in a highly dynamic or large-scale mobile environment. To address this crucial issue, we propose a DRL framework that capitalizes on prioritized sampling to speed up the learning process, thereby enabling rapid adaptation to the variations of the wireless environment. The proposed method is not only tailored to user mobility, but also to network variations due to device activation and deactivation. Numerical results demonstrate the effectiveness of our proposed algorithm, as it exhibits near-optimal performance, outperforming the benchmark schemes in terms of the guaranteed rate and total power consumption, with much faster convergence.