Data-Free Evaluation of User Contributions in Federated Learning
被引:0
作者:
Lv, Hongtao
论文数: 0引用数: 0
h-index: 0
机构:
Shanghai Jiao Tong Univ, Shanghai, Peoples R ChinaShanghai Jiao Tong Univ, Shanghai, Peoples R China
Lv, Hongtao
[1
]
Zheng, Zhenzhe
论文数: 0引用数: 0
h-index: 0
机构:
Shanghai Jiao Tong Univ, Shanghai, Peoples R ChinaShanghai Jiao Tong Univ, Shanghai, Peoples R China
Zheng, Zhenzhe
[1
]
Luo, Tie
论文数: 0引用数: 0
h-index: 0
机构:
Missouri Univ Sci & Technol, Rolla, MO 65409 USAShanghai Jiao Tong Univ, Shanghai, Peoples R China
Luo, Tie
[2
]
Wu, Fan
论文数: 0引用数: 0
h-index: 0
机构:
Shanghai Jiao Tong Univ, Shanghai, Peoples R ChinaShanghai Jiao Tong Univ, Shanghai, Peoples R China
Wu, Fan
[1
]
Tang, Shaojie
论文数: 0引用数: 0
h-index: 0
机构:
Univ Texas Dallas, Dallas, TX USAShanghai Jiao Tong Univ, Shanghai, Peoples R China
Tang, Shaojie
[3
]
Hua, Lifeng
论文数: 0引用数: 0
h-index: 0
机构:
Alibaba Grp, Hangzhou, Peoples R ChinaShanghai Jiao Tong Univ, Shanghai, Peoples R China
Hua, Lifeng
[4
]
Jie, Rongfei
论文数: 0引用数: 0
h-index: 0
机构:
Alibaba Grp, Hangzhou, Peoples R ChinaShanghai Jiao Tong Univ, Shanghai, Peoples R China
Jie, Rongfei
[4
]
Lv, Chengfei
论文数: 0引用数: 0
h-index: 0
机构:
Alibaba Grp, Hangzhou, Peoples R ChinaShanghai Jiao Tong Univ, Shanghai, Peoples R China
Lv, Chengfei
[4
]
机构:
[1] Shanghai Jiao Tong Univ, Shanghai, Peoples R China
[2] Missouri Univ Sci & Technol, Rolla, MO 65409 USA
[3] Univ Texas Dallas, Dallas, TX USA
[4] Alibaba Grp, Hangzhou, Peoples R China
来源:
2021 19TH INTERNATIONAL SYMPOSIUM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT)
|
2021年
基金:
美国国家科学基金会;
关键词:
Peer prediction;
correlated agreement;
D O I:
暂无
中图分类号:
TP [自动化技术、计算机技术];
学科分类号:
0812 ;
摘要:
Federated learning (FL) trains a machine learning model on mobile devices in a distributed manner using each device's private data and computing resources. A critical issues is to evaluate individual users' contributions so that (1) users' effort in model training can be compensated with proper incentives and (2) malicious and low-quality users can be detected and removed. The state-of-the-art solutions require a representative test dataset for the evaluation purpose, but such a dataset is often unavailable and hard to synthesize. In this paper, we propose a method called Pairwise Correlated Agreement (PCA) based on the idea of peer prediction to evaluate user contribution in FL without a test dataset. PCA achieves this using the statistical correlation of the model parameters uploaded by users. We then apply PCA to designing (1) a new federated learning algorithm called Fed-PCA, and (2) a new incentive mechanism that guarantees truthfulness. We evaluate the performance of PCA and Fed-PCA using the MNIST dataset and a large industrial product recommendation dataset. The results demonstrate that our Fed-PCA outperforms the canonical FedAvg algorithm and other baseline methods in accuracy, and at the same time, PCA effectively incentivizes users to behave truthfully.