FLSwitch: Towards Secure and Fast Model Aggregation for Federated Deep Learning with a Learning State-Aware Switch

Security and efficiency are two desirable properties of federated learning (FL). To enforce data security for FL participants, homomorphic encryption (HE) is widely adopted. However, existing solutions based on HE treat FL as a general computation task and apply HE protections indiscriminately at each step without considering FL computations' inherent characteristics, leading to unsatisfactory efficiency. In contrast, we find that the convergence process of FL generally consists of two phases, and the differences between these two phases can be exploited to improve the efficiency of secure FL solutions. In this paper, we propose a secure and fast FL solution named FLSwitch by tailoring different security protections for different learning phases. FLSwitch consists of three novel components, a new secure aggregation protocol based on the Pailliar HE and a residue number coding system outperforming the state-of-the-art HE-based solutions, a fast FL aggregation protocol with an extremely light overhead of learning on ciphertexts, and a learning state-aware decision model to switch between two protocols during an FL task. Since exploiting FL characteristics is orthogonal to optimizing HE techniques, FLSwitch can be applied to the existing HE-based FL solutions with cutting-edge optimizations, which could further boost secure FL efficiency.