Dynamic Black-Box Model Watermarking for Heterogeneous Federated Learning

Heterogeneous federated learning, as an innovative variant of federated learning, aims to break through the constraints of vanilla federated learning on the consistency of model architectures to better accommodate the heterogeneity in mobile computing scenarios. It introduces heterogeneous and personalized local models, which effectively accommodates the heterogeneous data distributions and hardware resource constraints of individual clients, and thus improves computation and communication efficiency. However, it poses a challenge to model ownership protection, as watermarks embedded in the global model are corrupted to varying degrees when they are migrated to a user's heterogeneous model and cannot continue to provide complete ownership protection in the local models. To tackle these issues, we propose a dynamic black-box model watermarking method for heterogeneous federated learning, PWFed. Specifically, we design an innovative dynamic watermark generation method which is based on generative adversarial network technology and is capable of generating watermark samples that are virtually indistinguishable from the original carriers. This approach effectively solves the limitation of the traditional black-box watermarking technique, which only considers static watermarks, and makes the generated watermarks significantly improved in terms of stealthiness and difficult to detect by potential model thieves, thus enhancing the robustness of the watermarks. In addition, we design two watermark embedding strategies with different granularities in the heterogeneous federated learning environment. During the watermark extraction and validation phase, PWFed accesses watermark samples claiming ownership of the model through an API interface and analyzes the differences between their output and the expected labels. Our experimental results show that PWFed achieves a 99.9% watermark verification rate with only a 0.1-4.8% sacrifice of main task accuracy on the CIFAR10 dataset.