Neural network quantization in federated learning at the edge

被引:34
作者
Tonellotto, Nicola [2 ]
Gotta, Alberto [1 ]
Nardini, Franco Maria [1 ]
Gadler, Daniele [4 ]
Silvestri, Fabrizio [3 ]
机构
[1] ISTI CNR, Pisa, Italy
[2] Univ Pisa, Pisa, Italy
[3] Sapienza Univ Rome, Rome, Italy
[4] ONE LOGIC GmbH, Frankfurt, Germany
基金
欧盟地平线“2020”;
关键词
Federated learning; Artificial neural networks; Quantization; Internet of Things; IOT; LSTM;
D O I
10.1016/j.ins.2021.06.039
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The massive amount of data collected in the Internet of Things (IoT) asks for effective, intel-ligent analytics. A recent trend supporting the use of Artificial Intelligence (AI) solutions in IoT domains is to move the computation closer to the data, i.e., from cloud-based services to edge devices. Federated learning (FL) is the primary approach adopted in this scenario to train AI-based solutions. In this work, we investigate the introduction of quantization tech-niques in FL to improve the efficiency of data exchange between edge servers and a cloud node. We focus on learning recurrent neural network models fed by edge data producers using the most widely adopted neural networks for time-series prediction. Experiments on public datasets show that the proposed quantization techniques in FL reduces up to 19x the volume of data exchanged between each edge server and a cloud node, with a min-imal impact of around 5% on the test loss of the final model. (c) 2021 Elsevier Inc. All rights reserved.
引用
收藏
页码:417 / 436
页数:20
相关论文
共 37 条
[1]   Social LSTM: Human Trajectory Prediction in Crowded Spaces [J].
Alahi, Alexandre ;
Goel, Kratarth ;
Ramanathan, Vignesh ;
Robicquet, Alexandre ;
Li Fei-Fei ;
Savarese, Silvio .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :961-971
[2]  
Alistarh D, 2017, ADV NEUR IN, V30
[3]  
Ardakani A., 2019, INT C LEARNING REPRE
[4]  
Bacco M, 2017, P IEEE INT C COMM IC, P1
[5]   Enhanced LSTM for Natural Language Inference [J].
Chen, Qian ;
Zhu, Xiaodan ;
Ling, Zhenhua ;
Wei, Si ;
Jiang, Hui ;
Inkpen, Diana .
PROCEEDINGS OF THE 55TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2017), VOL 1, 2017, :1657-1668
[6]  
Chen Xu., 2018, P ICLR
[7]  
Courbariaux M, 2015, ADV NEUR IN, V28
[8]   Adaptive greedy approximations [J].
Davis, G ;
Mallat, S ;
Avellaneda, M .
CONSTRUCTIVE APPROXIMATION, 1997, 13 (01) :57-98
[9]   Bonseyes AI Pipeline-Bringing AI to You: End-to-end integration of data, algorithms, and deployment tools [J].
De Prado, Miguel ;
Su, Jing ;
Saeed, Rabia ;
Keller, Lorenzo ;
Vallez, Noelia ;
Anderson, Andrew ;
Gregg, David ;
Benini, Luca ;
Llewellynn, Tim ;
Ouerhani, Nabil ;
Dahyot, Rozenn ;
Pazos, Nuria .
ACM TRANSACTIONS ON INTERNET OF THINGS, 2020, 1 (04)
[10]   A Survey of Communication Protocols for Internet of Things and Related Challenges of Fog and Cloud Computing Integration [J].
Dizdarevic, Jasenka ;
Carpio, Francisco ;
Jukan, Admela ;
Masip-Bruin, Xavi .
ACM COMPUTING SURVEYS, 2019, 51 (06)