DENSE: Data-Free One-Shot Federated Learning

被引:0
作者
Zhang, Jie [1 ,2 ,3 ]
Chen, Chen [1 ,3 ]
Li, Bo [2 ]
Lyu, Lingjuan [3 ]
Wu, Shuang
Ding, Shouhong [2 ]
Shen, Chunhua [1 ]
Wu, Chao [1 ]
机构
[1] Zhejiang Univ, Hangzhou, Peoples R China
[2] Tencent, Youtu Lab, Shenzhen, Peoples R China
[3] Sony AI, Boston, MA USA
来源
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022) | 2022年
基金
中国国家自然科学基金;
关键词
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round. Despite the low communication cost, existing one-shot FL methods are mostly impractical or face inherent limitations, e.g., a public dataset is required, clients' models are homogeneous, and additional data/model information need to be uploaded. To overcome these issues, we propose a novel two-stage Data-freE oNeShot federated lEarning (DENSE) framework, which trains the global model by a data generation stage and a model distillation stage. DENSE is a practical one-shot FL method that can be applied in reality due to the following advantages: (1) DENSE requires no additional information compared with other methods (except the model parameters) to be transferred between clients and the server; (2) DENSE does not require any auxiliary dataset for training; (3) DENSE considers model heterogeneity in FL, i.e., different clients can have different model architectures. Experiments on a variety of real-world datasets demonstrate the superiority of our method. For example, DENSE outperforms the best baseline method Fed-ADI by 5.08% on CIFAR10 dataset.
引用
收藏
页数:15
相关论文
共 64 条
[1]  
[Anonymous], 2021, P AAAI C ARTIFICIAL
[2]  
Cao K., 2019, Learning imbalanced datasets with label-distribution-aware margin loss
[3]   Data-Free Learning of Student Networks [J].
Chen, Hanting ;
Wang, Yunhe ;
Xu, Chang ;
Yang, Zhaohui ;
Liu, Chuanjian ;
Shi, Boxin ;
Xu, Chunjing ;
Xu, Chao ;
Tian, Qi .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :3513-3521
[4]  
Chen Hong-You, 2021, Fedbe: Making bayesian model ensemble applicable to federated learning
[5]  
Chen Zehui, 2022, EUR C COMP VIS
[6]   Towards Practical Certifiable Patch Defense with Vision Transformer [J].
Chen, Zhaoyu ;
Li, Bo ;
Xu, Jianghe ;
Wu, Shuang ;
Ding, Shouhong ;
Zhang, Wenqiang .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :15127-15137
[7]   Personalized Retrogress-Resilient Framework for Real-World Medical Federated Learning [J].
Chen, Zhen ;
Zhu, Meilu ;
Yang, Chen ;
Yuan, Yixuan .
MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION - MICCAI 2021, PT III, 2021, 12903 :347-356
[8]  
Dennis Don Kurian, 2021, Heterogeneity for the win: One-shot federated clustering
[9]   Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation [J].
Dong, Jiahua ;
Cong, Yang ;
Sun, Gan ;
Fang, Zhen ;
Ding, Zhengming .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2024, 46 (03) :1664-1681
[10]   What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation [J].
Dong, Jiahua ;
Cong, Yang ;
Sun, Gan ;
Zhong, Bineng ;
Xu, Xiaowei .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :4022-4031