Rate Control for RIS-Empowered Multi-Cell Dual-Connectivity HetNets: A Distributed Multi-Task DRL Approach

Heterogeneous wireless networks (HetNets), where networks are deployed with ultra-dense small cells (SCs), is one of the main enabling technologies for future wireless networks. In such networks, signals are vulnerable to severe blockage, interference, and intermittent connectivity. This can be largely overcome using the emerging Reconfigurable Intelligent Surface (RIS) technology that can enhance HetNets performance by controlling the propagation environment. However, jointly optimizing the parameters of base stations' (BSs') active beamforming and RISs' passive beamforming is a major challenge in RIS-empowered HetNets. In this paper, we investigate the issue of rate control in RIS-empowered multi-cell multiple-input single-output (MISO) HetNets via joint users' equipment (UEs) rate fairness and SCs rate load balancing. We assume RIS-assisted SC BSs at mmWave underlying a RIS-assisted macrocell (MC) BS at sub-6GHz serving dual-connectivity UEs that can concurrently connect to the MC BS and a single SC BS. Then, we formulate an optimization problem whose objective is to jointly optimize the active transmit beamforming vectors of the MC and SCs BSs on the one hand and the passive beamforming vectors of the MC and SCs RISs on the other hand. Due to the high non-convexity and complexity of the formulated problem, we propose a novel distributed Deep Deterministic Policy Gradient (DDPG)-based multi-task deep reinforcement learning (MTDRL) scheme to solve the problem and learn network dynamics. Through deliberate definitions of MTDRL agent's tasks and their corresponding main elements, we demonstrate via simulations that our proposed scheme guarantees a fair distribution of rates within UEs and SCs. In addition, we quantify the robustness of our proposed MTDRL scheme compared with some benchmarks in terms of convergence speed and utility values.