Privacy-Preserving Federated Reinforcement Learning for Popularity-Assisted Edge Caching

In this paper, we investigate the problem of edge caching (EC) optimization in a multi-user privacy-preserving mobile edge computing (MEC) system. The time-varying content popularity is considered and the primary objective is to maximize the EC hit rate on each caching entity in the distributed network. To this end, we introduce the concept of local and global popularities and cast the time-varying local popularities as model-free Markov chains. Next, an unsupervised recurrent federated learning (URFL) algorithm is proposed to predict the popularities while achieving privacy-preserving goal. The underlying distributed optimization problem is then reformulated as a distributed Markov decision process and solved by the privacy-preserving distributed deep deterministic policy gradient algorithm incorporating the URFL algorithm. Simulation results demonstrate the superiority of the proposed scheme in terms of prediction error and hit rate over the baseline methods.