A PATE-based Approach for Training Graph Neural Networks under Label Differential Privacy

As a standard solution to the problem of private deep learning, differential privacy (DP) is widely used in graph neural networks (GNNs) to protect sensitive information about the input graph data. However, most existing DP algorithms for GNNs protect the privacy of every attribute for each node. This results in the need for injecting a large amount of noise, making these methods significantly underperform their nonprivate counterparts. We argue that in some practical scenarios, node labels serve as the only or the most sensitive attribute, where label differential privacy, a more fine-grained notion of differential privacy that only protects the labels is more appropriate. To better capture these scenarios and improve the trade-off between data privacy and model accuracy, we propose a novel method of training GNNs under label differential privacy. Instead of naively adding noise to the node labels before training the GNN, our method follows the strategy of Private Aggregation of Teacher Ensembles (PATE) to generate differentially private node labels with both high accuracy and strong privacy guarantee. We also propose a label denoising module that takes advantage of the graph structure to further improve the accuracy of the trained model. Additionally, our method is model-agnostic, making it applicable to any GNN architecture. We evaluate its performance on two commonly used benchmark datasets and demonstrate its capability to learn high-performance models while ensuring privacy.