FAST DECENTRALIZED LEARNING VIA HYBRID CONSENSUS ADMM

The Alternating Directions Methods of Multipliers (ADMM) has witnessed a resurgence of interest over the past few years fueled by the ever increasing demand for scalable optimization techniques to tackle real-world statistical learning problems. However, despite its success in several application settings the applicability of the traditional centralized ADMM is limited by its communication requirement to a global fusion center, which might not be always feasible. Its decentralized variant D-CADMM, on the other hand, while it alleviates this need, it does so at the expense of significantly slower convergence in cases of adverse underlying network topologies. To address the aforementioned limitations, in this work we consider the presence of multiple fusion centers and we propose a unifying framework that allows leveraging the structure of the communication network to accelerate the decentralized ADMM even in cases where it is not practical to resort to its fully centralized counterpart. We prove the linear convergence rate of the proposed approach and we verify its promising performance by carrying out numerical tests on both real and synthetic networks.