Deep Reinforcement Learning for Distributed Dynamic MISO Downlink-Beamforming Coordination

We consider a homogeneous cellular network where a multi-antenna base station (BS) in each cell transmits messages to its intended user over a common frequency band. To improve the system capacity of this multi-cell multi-input single-output (MISO) interference channel, one of the state-of-the-art algorithms, namely, downlink-beamforming coordination, allows all BSs to cooperate with one another to mitigate the effect of inter-cell interference. However, most existing algorithms are suboptimal and impractical in a dynamic wireless environment, due to the high computational complexity and the overhead involved in collecting global channel state information (CSI). In this study, we exploit deep reinforcement learning (DRL) and propose a distributed dynamic downlink-beamforming coordination (DDBC) method with partial observability of the CSI. Each BS is able to train its own deep Q-network and employs appropriate beamformer depending on its environment, which is observed through a designed limited-information exchange protocol. The simulation results show that the proposed DRL-based DDBC method, with a considerably lower system overhead, achieves a system capacity that is very close to that of the fractional programming algorithm with global and instantaneous CSI measurements. In addition, this work demonstrates the potential of utilizing DRL to solve DDBC problems in a more practical manner.