Toward Communication-Learning Trade-Off for Federated Learning at the Network Edge

被引:14
作者
Ren, Jianyang [1 ]
Ni, Wanli [1 ]
Tian, Hui [1 ]
机构
[1] Beijing Univ Posts & Telecommun, State Key Lab Networking & Switching Technol, Beijing 100876, Peoples R China
基金
国家重点研发计划;
关键词
Convergence; Training; Costs; Upper bound; Error analysis; Data models; Collaborative work; Federated learning; network pruning; convergence analysis; bandwidth allocation;
D O I
10.1109/LCOMM.2022.3174295
中图分类号
TN [电子技术、通信技术];
学科分类号
0809 ;
摘要
In this letter, we study a wireless federated learning (FL) system where network pruning is applied to local users with limited resources. Although pruning is beneficial to reduce FL latency, it also deteriorates learning performance due to the information loss. Thus, a trade-off problem between communication and learning is raised. To address this challenge, we quantify the effects of network pruning and packet error on the learning performance by deriving the convergence rate of FL with a non-convex loss function. Then, closed-form solutions for pruning control and bandwidth allocation are proposed to minimize the weighted sum of FL latency and FL performance. Finally, numerical results demonstrate that i) our proposed solution can outperform benchmarks in terms of cost reduction and accuracy guarantee, and ii) a higher pruning rate would bring less communication overhead but also worsen FL accuracy, which is consistent with our theoretical analysis.
引用
收藏
页码:1858 / 1862
页数:5
相关论文
共 11 条
[1]   A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks [J].
Chen, Mingzhe ;
Yang, Zhaohui ;
Saad, Walid ;
Yin, Changchuan ;
Poor, H. Vincent ;
Cui, Shuguang .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2021, 20 (01) :269-283
[2]   Federated Learning Over Wireless Networks: Convergence Analysis and Resource Allocation [J].
Dinh, Canh T. ;
Tran, Nguyen H. ;
Nguyen, Minh N. H. ;
Hong, Choong Seon ;
Bao, Wei ;
Zomaya, Albert Y. ;
Gramoli, Vincent .
IEEE-ACM TRANSACTIONS ON NETWORKING, 2021, 29 (01) :398-409
[3]  
Jiang Y., 2019, arXiv
[4]   Novel three-dimensional multi-objective numerical modeling for hot strip tandem rolling [J].
Li, Lianjie ;
Li, Jianxin ;
Xie, Haibo ;
Liu, Hongqiang ;
Sun, Li ;
Liu, Tianwu ;
Liu, Xu ;
Shi, Kexin ;
Jiang, Zhengyi .
INTERNATIONAL JOURNAL OF MATERIAL FORMING, 2021, 14 (05) :989-1004
[5]   Adaptive Network Pruning for Wireless Federated Learning [J].
Liu, Shengli ;
Yu, Guanding ;
Yin, Rui ;
Yuan, Jiantao .
IEEE WIRELESS COMMUNICATIONS LETTERS, 2021, 10 (07) :1572-1576
[6]   Cost-Effective Federated Learning Design [J].
Luo, Bing ;
Li, Xiang ;
Wang, Shiqiang ;
Huang, Jianwei ;
Tassiula, Leandros .
IEEE CONFERENCE ON COMPUTER COMMUNICATIONS (IEEE INFOCOM 2021), 2021,
[7]   Importance Estimation for Neural Network Pruning [J].
Molchanov, Pavlo ;
Mallya, Arun ;
Tyree, Stephen ;
Frosio, Iuri ;
Kautz, Jan .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :11256-11264
[8]  
Ren J., 2021, 2021 IEEE INT C COMM, P1
[9]   A General Upper Bound to Evaluate Packet Error Rate over Quasi-Static Fading Channels [J].
Xi, Yong ;
Burr, Alister ;
Wei, JiBo ;
Grace, David .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2011, 10 (05) :1373-1377
[10]   Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective [J].
Xu, Jie ;
Wang, Heqiang .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2021, 20 (02) :1188-1200