Robust Clustered Federated Learning with Bootstrap Median-of-Means

Federated learning (FL) is a new machine learning paradigm to collaboratively learn an intelligent model across many clients without uploading local data to the server. Non-IID data across clients is a major challenge for the FL system because its inherited distributed machine learning framework is designed for the scenario of IID data across clients. Clustered FL is a type of FL method to solve non-IID challenges using a client clustering method in the FL context. However, existing clustered FL methods suffer the challenge of processing client-wise outliers which could be produced by minority clients with abnormal behaviour patterns or be derived from malicious clients. This paper is to propose a novel Federated learning framework with Robust Clustering (FedRoC) to tackle client-wise outliers in the FL system. Specifically, we will develop a robust federated aggregation operator using a bootstrap median-of-means mechanism that can produce a higher breakdown point to tolerate a larger proportion of outliers. We formulate the proposed FL framework into a bi-level optimization problem, and then a stochastic expectation-maximization method is adopted to solve the optimization problem in an alternative updating manner by considering EM steps and distributed computing simultaneously. The experiments on three benchmark datasets have demonstrated the effectiveness of the proposed method that outperforms other baseline methods in terms of evaluation criteria.