Robust decentralized federated learning for heterogeneous and non-ideal networks

This study explores the challenges in non-ideal and heterogeneous distributed communication systems, where each device performs multiple iterations and exchanges incomplete information with its neighboring devices. We first focus on the impact of channel noise in decentralized federated learning (DFL), wherein the reception of signals by devices is susceptible to interference from noise. We then analyze the adverse effects of channel noise on the convergence and generalization of DFL. To mitigate the impact of channel noise, we propose a novel DFL with global momentum algorithm, called DFLGM, designed to adaptively control channel noise. Specifically, each device actively shares differential models with neighbors through a noisy channel to establish global consensus. Following information interchange, the DFLGM algorithm utilizes global momentum updates to preserve historical information and mitigate the impact of communication noise. We present the convergence results of DFLGM in the context of general non-convex and heterogeneous optimization problems. Extensive experiments and comparisons with the state-of-the-art methods demonstrate the effectiveness of the DFLGM algorithm.