Distributed Noncoherent Joint Transmission Based on Multi-Agent Reinforcement Learning for Dense Small Cell Networks

In dense small cell networks, the coordinated multi-point noncoherent joint transmission (JT) is a key technique to mitigate inter-cell interference and enhance network capacity. However, the capacity-maximizing power control and beamforming problem subject to a total transmit power constraint at each individual small cell base station (BS) is inherently nonconvex and NP-hard. To solve this problem, most existing algorithms require global channel state information (CSI) and a lot of computations, which are infeasible and impractical in dynamic wireless networks with limited computing power and link capacity. In this paper, we characterize a low-dimensional solution structure for the power control of the sum-rate maximization problem in time division duplex (TDD) dense small cell networks. Taking advantage of this low-dimensional structure, a distributed noncoherent JT scheme based on multi-agent reinforcement learning (MARL) is proposed to maximize the sum-rate of the dense small cell networks with reduced information overhead. In the proposed scheme, each BS acts as an agent and makes decisions locally. It is proved that the optimal sum-rate can be achieved for single-transmit-antenna BSs by using the proposed scheme. Compared to the best method presently known, the proposed scheme achieves a similar sum-rate with considerably lower computational complexity and information overhead, which makes it more appealing for practical implementations.