Sugar-coated poison defense on deepfake face-swapping attacks

The deployment of deepfake face-swapping technology has matured, becoming widespread on the Internet. The misuse of this technology raises significant concerns for application security and privacy. To counter deepfake threats, we propose a sugar-coated poison defense targeting the latent vectors of generative models. This strategy aims to impact visual effects without substantially increasing reconstruction loss. We establish metrics for visual effects and reconstruction loss to assess perturbation effects on latent vectors, emphasizing those with the most significant impact on visual effects while minimizing reconstruction loss. Our approach begins by utilizing a facial feature extraction model to convert faces into latent representations. We then introduce two latent selection methods: 1) shap-based latent selection using a linear regression model for approximation, and 2) grid search latent selection employing heuristics of adversarial attacks. These methods pinpoint vectors that, when perturbed, can increase face landmark distances while maintaining low mean square errors, commonly used as the optimization metric in deepfake reconstruction models. We apply inconsistent perturbations to selected latent vectors in video frames, acting as sugar-coated poison for deepfake face-swapping applications. Preliminary results demonstrate that these perturbations can be applied to individual videos, resulting in low reconstruction loss. Importantly, they induce measurable consistency reduction in deepfake videos, making them more discernible and accessible to identify.