FedGroup-Prune: IoT Device Amicable and Training-Efficient Federated Learning via Combined Group Lasso Sparse Model Pruning

被引:0
作者
Chen, Ziyao [1 ]
Peng, Jialiang [1 ]
Kang, Jiawen [2 ]
Niyato, Dusit [3 ]
机构
[1] Heilongjiang Univ, Sch Comp & Big Data, Harbin 150080, Peoples R China
[2] Guangdong Univ Technol, Sch Automat, Guangzhou 510006, Peoples R China
[3] Nanyang Technol Univ, Sch Comp Sci & Engn, Singapore, Singapore
来源
IEEE INTERNET OF THINGS JOURNAL | 2024年 / 11卷 / 24期
基金
新加坡国家研究基金会;
关键词
Computational modeling; Training; Data models; Neurons; Performance evaluation; Federated learning (FL); Group Lasso; Internet of Things (IoT); model pruning;
D O I
10.1109/JIOT.2024.3457871
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Federated learning (FL) has emerged as a crucial approach in the realm of distributed machine learning, providing a framework for training models on decentralized data while preserving data privacy. This paradigm has established itself as an effective solution for deploying artificial intelligence technology in scenarios associated with the Internet of Things (IoT). Despite its potential, FL faces encounters several challenges, particularly the limited computational and communication capabilities of some local clients, which can hinder further advancement. Such constraints limit the effective implementation and utilization of deep neural networks (DNNs) with numerous parameters on IoT devices. Our study tackles this issue by utilizing Group Lasso for model sparsification and pruning, aimed at lowering the computational and communication demands on IoT devices. Moreover, this article proposes a Group Lasso-enabled FL model pruning strategy specifically tailored for IoT, designed to reduce the size of model parameters, and provides theoretical guarantees of FL convergence. Empirical analysis across multiple models and data sets demonstrates that our method effectively halved the parameters in fully connected layers during federated training. This substantial reduction is achieved with minimal impact on accuracy, thus preserving the integrity of model performance and providing a competitive edge over existing methodologies.
引用
收藏
页码:40921 / 40932
页数:12
相关论文
共 45 条
[1]   Review of deep learning: concepts, CNN architectures, challenges, applications, future directions [J].
Alzubaidi, Laith ;
Zhang, Jinglan ;
Humaidi, Amjad J. ;
Al-Dujaili, Ayad ;
Duan, Ye ;
Al-Shamma, Omran ;
Santamaria, J. ;
Fadhel, Mohammed A. ;
Al-Amidie, Muthana ;
Farhan, Laith .
JOURNAL OF BIG DATA, 2021, 8 (01)
[2]  
Brown T., 2020, PROC 34 INT C NEURAL, V33, P1877, DOI [10.48550/ARXIV.2005.14165, DOI 10.48550/ARXIV.2005.14165]
[3]  
Caldas S, 2019, Arxiv, DOI arXiv:1812.07210
[4]  
Cheng HR, 2024, Arxiv, DOI arXiv:2308.06767
[5]  
Dosovitskiy A, 2021, Arxiv, DOI arXiv:2010.11929
[6]  
Guo YW, 2016, ADV NEUR IN, V29
[7]   Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach [J].
Han, Pengchao ;
Wang, Shiqiang ;
Leung, Kin K. .
2020 IEEE 40TH INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS (ICDCS), 2020, :300-310
[8]  
Han S, 2016, Arxiv, DOI arXiv:1510.00149
[9]  
Han S, 2015, ADV NEUR IN, V28
[10]   EIE: Efficient Inference Engine on Compressed Deep Neural Network [J].
Han, Song ;
Liu, Xingyu ;
Mao, Huizi ;
Pu, Jing ;
Pedram, Ardavan ;
Horowitz, Mark A. ;
Dally, William J. .
2016 ACM/IEEE 43RD ANNUAL INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE (ISCA), 2016, :243-254