Joint UAV Deployment and Resource Allocation: A Personalized Federated Deep Reinforcement Learning Approach

Unmanned aerial vehicles (UAVs) are capable of serving as aerial base stations (BSs) for providing dynamic coverage and connectivity extension for the sixth-generation (6G) wireless networks. While flexibility is provided, the deployment of the UAV swarms and the associated resource allocation become rather challenging due to the dynamic nature of UAVs and difficulty in obtaining global user information. In this article, we propose an adaptive and flexible joint UAV deployment and resource allocation (JUDRA) scheme by exploiting personalized federated deep reinforcement learning, called PFRL, with aim to maximize the long-term network throughput while enforcing user privacy and adapting to time-varying network states. To allow UAVs to make real-time decisions on resource allocation and position adjustment based on local observations while achieving a global optimal solution, a deep reinforcement learning (DRL) algorithm is adopted in the federated learning framework in PFRL. Specifically, we use DRL to train a local model and a personalized model on UAVs, and employ a two-level parameter aggregation scheme on a leading UAV to form a global model. The personalized model can adapt to changing environments, while exploiting the generalization of global model to accelerate the learning convergence. Numerical results show that the proposed PFRL scheme can achieve significant performance gain in terms of network throughput and convergence in comparison with some state-of-art solutions.