Heterogeneous Federated Learning Based on Graph Hypernetwork

Federated learning is a distributed machine learning framework over a large number of possible clients without data leakage. However, most federated learning methods are limited to clients with isomorphic network architectures. This restricts heterogeneous clients equipped with different computation and communication resources. To address this issue, this paper introduces a new heterogeneous federated learning method called heterogeneous federated graph hyperNetwork (hFedGHN), which takes the clients as a graph to integrate both structure and content information for deep latent representation learning and generates heterogeneous model for each client. HFedGHN can share effective parameters across clients, and maintain the capacity to generate unique and diverse personal models. Moreover, hFedGHN can be regarded as a process to learn a meta-model for federated learning, therefore it has better generalization ability to novel clients. As a group of novel clients are added dynamically, they can get acceptable performance after just one round of communication. With extensive experiments on MNIST, CIFAR-10, and CIFAR-100, the results demonstrate the superiority of hFedGHN.