RDP-GAN: A Renyi-Differential Privacy Based Generative Adversarial Network

Generative adversarial networks (GANs) have attracted increasing attention recently owing to their impressive abilities to generate realistic samples with high privacy protection. Without directly interacting with training examples, the generative model can be used to estimate the underlying distribution of an original dataset while the discriminator can examine model quality of the generated samples by comparing the label values with training examples. In considering privacy issues in GANS, existing works focus on perturbing the parameters and analyzing the corresponding privacy protection capability, and the parameters are not directly exchanged between the generator and discriminator in GANs. Thus, in this work, we propose a Renyi-differentially private-GAN (RDP-GAN), which achieves differential privacy (DP) in a GAN by carefully adding random Gaussian noise to the value of the exchanged loss function during training. Moreover, we derive analytical results characterizing the total privacy loss under the subsampling method and cumulative iterations, which show its effectiveness for the privacy budget allocation. In addition, in order to mitigate the negative impact of injecting noises, we enhance the proposed algorithm by adding an adaptive noise tuning step, which will change the amount of added noise according to the testing accuracy. Through extensive experimental results, we verify that the proposed algorithm can achieve a better privacy level while producing high-quality samples compared with a benchmark DP-GAN scheme based on noise perturbation on training gradients.