Federated Learning with Personalized Differential Privacy Combining Client Selection

Federated Learning(FL) enables clients to construct a global model collaboratively while protecting the security and privacy of clients' datasets. Differential privacy(DP) is a common method to protect clients' data privacy in FL. However, most of existing works assume the privacy needs of all clients are the same, which is rare in reality, making it difficult to apply their approaches to realistic scenarios. What's more, existing works considering DP protection are not combined with client selection, which will result in clients with the same dataset quality but different added noise introduced by DP mechanism having the same chance of being selected, which is likely to harm model performance. In this work, we consider personalized DP(PDP) setting, i.e. each client has personal privacy need. We conduct a convergence analysis of FL with PDP and client selection. We find that for fast convergence and small bias, we prefer clients with high local loss and small added noise. Moreover, the influence of noise on the later stage of training is greater than that in the earlier stage. According to the analysis result, we propose a novel mechanism, containing a PDP mechanism combining sampling for small added noise while meeting the privacy needs of clients and a client selection mechanism with a novel metric score which considers the local loss and privacy needs of clients at the same time. Our experiments demonstrate that the performance of our mechanism is better than the baseline mechanism.