Evil vs evil: using adversarial examples to against backdoor attack in federated learning

As a distributed learning paradigm, federated learning (FL) has shown great success in aggregating information from different clients to train a shared global model. Unfortunately, by uploading carefully crafted updated models, a malicious client can embed a backdoor into the global model during FL's training. Numerous secure aggregation strategies and robust training protocols have been proposed to defend FL against backdoor attacks. However, they are still challenged, either being bypassed by adaptive attacks or sacrificing the main task performance of FL. By conducting empirical studies of backdoor attacks in FL, we gain an interesting insight that adversarial perturbations can activate backdoors in backdoor models. Consequently, behavior differences of models fed by adversarial examples are compared for backdoor update detection. We propose a novel FL backdoor defense method using adversarial examples, denoted as Evil vs Evil (EVE). Specifically, a small data set of clean examples for FL's main task training is collected in the sever for adversarial examples generation. By observing the behavior of updated models under the adversarial examples, EVE uses a clustering algorithm to select benign models and to exclude the other models, without any loss of the main task performance of FL itself. Extensive evaluations across four data sets and the corresponding DNNs demonstrate the state-of-the-art (SOTA) defense performance of EVE compared with five baselines. In particular, EVE under 40% of malicious clients can reduce the attack success rate from 99% to 1%. In addition, we verify that EVE is still robust under the adaptive attacks. EVE is open sourced to facilitate future research.