A Deep Reinforcement Learning Approach to Two-Timescale Transmission for RIS-Aided Multiuser MISO systems

Reconfigurable intelligent surface (RIS) has drawn great attention recently as a promising technology for future wireless networks. In this letter, considering the two-timescale transmission protocol, we investigate the joint design of the transmit beamforming at the base station (BS) with instantaneous channel state information (CSI) and the RIS phase shifts with statistical CSI. Due to the large number of RIS elements, this design issue usually suffers from high computational complexity. To resolve the non-convexity issue with low complexity, we propose a novel deep reinforcement learning (DRL) framework, which contains two agents applying proximal policy optimization (PPO) based algorithm. Experiment results demonstrate that the proposed algorithm has comparable spectral efficiency performance to the state-of-the-art methods with substantially reduced computational delay.