Byzantine-resilient Bilevel Federated Learning

To tackle new learning criteria such as robustness and automation, many machine learning problems today involve nested structures and are thus often formulated as bilevel learning problems. To leverage data from multiple data owners, we consider bilevel learning in the federated setting in the presence of Byzantine clients. We propose a byzantine-resilient bilevel federated optimization algorithm that we call BILANTINE and provide a theoretical analysis establishing its convergence rate. We empirically demonstrate our method's effectiveness on the data reweighting task under various attacks and show that it can achieve nearly the same performance as the system would have achieved in the absence of attacks. To the best of our knowledge, this is the first empirical and theoretical study of federated bilevel optimization under Byzantine attacks.