Balancing Privacy and Performance: A Differential Privacy Approach in Federated Learning

Federated learning (FL), a decentralized approach to machine learning, facilitates model training across multiple devices, ensuring data privacy. However, achieving a delicate privacy preservation-model convergence balance remains a major problem. Understanding how different hyperparameters affect this balance is crucial for optimizing FL systems. This article examines the impact of various hyperparameters, like the privacy budget (& varepsilon;), clipping norm (C), and the number of randomly chosen clients (K) per communication round. Through a comprehensive set of experiments, we compare training scenarios under both independent and identically distributed (IID) and non-independent and identically distributed (Non-IID) data settings. Our findings reveal that the combination of & varepsilon; and C significantly influences the global noise variance, affecting the model's performance in both IID and Non-IID scenarios. Stricter privacy conditions lead to fluctuating non-converging loss behavior, particularly in Non-IID settings. We consider the number of clients (K) and its impact on the loss fluctuations and the convergence improvement, particularly under strict privacy measures. Thus, Non-IID settings are more responsive to stricter privacy regulations; yet, with a higher client interaction volume, they also can offer better convergence. Collectively, knowledge of the privacy-preserving approach in FL has been extended and useful suggestions towards an ideal privacy-convergence balance were achieved.