Decentralized Joint Pilot and Data Power Control Based on Deep Reinforcement Learning for the Uplink of Cell-Free Systems

While the problem of jointly controlling the pilot-and-data power in cell-based systems has been extensively studied, this problem is difficult to solve in cell-free systems due to two reasons. First, both the large- and small-scale fading are markedly different between a served user and the multiple serving access points. Second, due to the user-centric architecture, there is a need for decentralized algorithms that scale well in the cell-free environment. In this work, we study the impact of joint pilot-and-data power control and receive filter design in the uplink of cell-free systems. The problem is formulated as optimization tasks considering two different objectives: 1) maximization of the minimum spectral efficiency (SE) and 2) maximization of the total SE. Since these problems are non-convex, we resort to successive convex approximation and geometric programming to obtain a local optimal centralized solution for benchmarking purposes. We also propose a decentralized solution based on actor-critic deep reinforcement learning, in which each user acts as an agent to locally obtain the best policy relying on minimum information exchange. Practical signaling aspects are provided for such a decentralized solution. Finally, numerical results indicate that the decentralized solution performs very close to the centralized one and outperforms state-of-the-art algorithms in terms of minimum SE and total system SE.