Robust Federated Learning for Unreliable and Resource-Limited Wireless Networks

被引:4
作者
Chen, Zhixiong [1 ]
Yi, Wenqiang [2 ]
Liu, Yuanwei [1 ]
Nallanathan, Arumugam [1 ]
机构
[1] Queen Mary Univ London, Sch Elect Engn & Comp Sci, London E1 4NS, England
[2] Univ Essex, Sch Comp Sci & Elect Engn, Colchester CO4, England
基金
英国工程与自然科学研究理事会;
关键词
Resource management; Convergence; Wireless networks; Performance evaluation; Servers; Power control; Training; Device scheduling; federated learning; resource allocation; unreliable transmission; CONVERGENCE;
D O I
10.1109/TWC.2024.3366393
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Federated learning (FL) is an efficient and privacy-preserving distributed learning paradigm that enables massive edge devices to train machine learning models collaboratively. Although various communication schemes have been proposed to expedite the FL process in resource-limited wireless networks, the unreliable nature of wireless channels was less explored. In this work, we propose a novel FL framework, namely FL with gradient recycling (FL-GR), which recycles the historical gradients of unscheduled and transmission-failure devices to improve the learning performance of FL. To reduce the hardware requirements for implementing FL-GR in the practical network, we develop a memory-friendly FL-GR that is equivalent to FL-GR but requires low memory of the edge server. We then theoretically analyze how the wireless network parameters affect the convergence bound of FL-GR, revealing that minimizing the average square of local gradients' staleness (AS-GS) helps improve the learning performance. Based on this, we formulate a joint device scheduling, resource allocation and power control optimization problem to minimize the AS-GS for global loss minimization. To solve the problem, we first derive the optimal power control policy for devices and transform the AS-GS minimization problem into a bipartite graph matching problem. Through detailed analysis, we further transform the bipartite matching problem into an equivalent linear program which is convenient to solve. Extensive simulation results on three real-world datasets (i.e., MNIST, CIFAR-10, and CIFAR-100) verified the efficacy of the proposed methods. Compared to the FL algorithms without gradient recycling, FL-GR is able to achieve higher accuracy and fast convergence speed. In addition, the proposed device scheduling and resource allocation algorithm also outperforms the benchmarks in accuracy and convergence speed.
引用
收藏
页码:9793 / 9809
页数:17
相关论文
共 40 条
[1]  
Abbasnejad E., 2018, Deep Lipschitz Networks and Dudley GANs
[2]   Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edge [J].
Amiri, Mohammad Mohammadi ;
Gunduz, Deniz ;
Kulkarni, Sanjeev R. ;
Poor, H. Vincent .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2021, 20 (06) :3643-3658
[3]   Federated Learning Over Wireless Fading Channels [J].
Amiri, Mohammad Mohammadi ;
Gunduz, Deniz .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2020, 19 (05) :3546-3557
[4]   Federated Learning Over Wireless IoT Networks With Optimized Communication and Resources [J].
Chen, Hao ;
Huang, Shaocheng ;
Zhang, Deyou ;
Xiao, Ming ;
Skoglund, Mikael ;
Poor, H. Vincent .
IEEE INTERNET OF THINGS JOURNAL, 2022, 9 (17) :16592-16605
[5]   Convergence Time Optimization for Federated Learning Over Wireless Networks [J].
Chen, Mingzhe ;
Poor, H. Vincent ;
Saad, Walid ;
Cui, Shuguang .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2021, 20 (04) :2457-2471
[6]   A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks [J].
Chen, Mingzhe ;
Yang, Zhaohui ;
Saad, Walid ;
Yin, Changchuan ;
Poor, H. Vincent ;
Cui, Shuguang .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2021, 20 (01) :269-283
[7]   Exploring Representativity in Device Scheduling for Wireless Federated Learning [J].
Chen, Zhixiong ;
Yi, Wenqiang ;
Nallanathan, Arumugam .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2024, 23 (01) :720-735
[8]   Convergence Analysis for Wireless Federated Learning with Gradient Recycling [J].
Chen, Zhixiong ;
Yi, Wenqiang ;
Liu, Yuanwei ;
Nallanathan, Arumugam .
2023 INTERNATIONAL WIRELESS COMMUNICATIONS AND MOBILE COMPUTING, IWCMC, 2023, :1232-1237
[9]   Knowledge-Aided Federated Learning for Energy-Limited Wireless Networks [J].
Chen, Zhixiong ;
Yi, Wenqiang ;
Liu, Yuanwei ;
Nallanathan, Arumugam .
IEEE TRANSACTIONS ON COMMUNICATIONS, 2023, 71 (06) :3368-3386
[10]  
Chen ZX, 2022, Arxiv, DOI arXiv:2204.09746