FakeTracer: Catching Face-Swap DeepFakes via Implanting Traces in Training

被引:0
作者
Sun, Pu [1 ]
Qi, Honggang [1 ]
Li, Yuezun [2 ]
Lyu, Siwei [3 ]
机构
[1] Univ Chinese Acad Sci, Beijing 101408, Peoples R China
[2] Ocean Univ China, Coll Comp Sci & Technol, Qingdao 266005, Shandong, Peoples R China
[3] SUNY Buffalo, Amherst, NY 14068 USA
基金
中国博士后科学基金;
关键词
DeepFake; multimedia forensics; proactive defence;
D O I
10.1109/TETC.2024.3386960
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Face-swap DeepFake is an emerging AI-based face forgery technique that can replace the original face in a video with a generated face of the target identity while retaining consistent facial attributes such as expression and orientation. Due to the high privacy of faces, the misuse of this technique can raise severe social concerns, drawing tremendous attention to defend against DeepFakes recently. In this article, we describe a new proactive defense method called FakeTracer to expose face-swap DeepFakes via implanting traces in training. Compared to general face-synthesis DeepFake, the face-swap DeepFake is more complex as it involves identity change, is subjected to the encoding-decoding process, and is trained unsupervised, increasing the difficulty of implanting traces into the training phase. To effectively defend against face-swap DeepFake, we design two types of traces, sustainable trace (STrace) and erasable trace (ETrace), to be added to training faces. During the training, these manipulated faces affect the learning of the face-swap DeepFake model, enabling it to generate faces that only contain sustainable traces. In light of these two traces, our method can effectively expose DeepFakes by identifying them. Extensive experiments corroborate the efficacy of our method on defending against face-swap DeepFake.
引用
收藏
页码:134 / 146
页数:13
相关论文
共 43 条
[11]   Towards Generating Ultra-High Resolution Talking-Face Videos with Lip synchronization [J].
Gupta, Anchit ;
Mukhopadhyay, Rudrabha ;
Balachandra, Sindhu ;
Khan, Faizan Farooq ;
Namboodiri, Vinay P. ;
Jawahar, C. V. .
2023 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2023, :5198-5207
[12]  
Hernandez-Ortega J, 2020, Arxiv, DOI [arXiv:2010.00400, DOI 10.48550/ARXIV.2010.00400, 10.48550/arXiv.2010.00400]
[13]  
Karras T., 2018, P IEEE INT C LEARN R, P1, Patent No. 171010196
[14]   Analyzing and Improving the Image Quality of StyleGAN [J].
Karras, Tero ;
Laine, Samuli ;
Aittala, Miika ;
Hellsten, Janne ;
Lehtinen, Jaakko ;
Aila, Timo .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), 2020, :8107-8116
[15]   A Style-Based Generator Architecture for Generative Adversarial Networks [J].
Karras, Tero ;
Laine, Samuli ;
Aila, Timo .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :4396-4405
[16]  
Kingma D.P., 2013, 2 INT C LEARN REPR I, DOI [DOI 10.61603/CEAS.V2I1.33, DOI 10.48550/ARXIV.1312.6114, 10.61603/ceas.v2i1.33]
[17]   Identification of deep network generated images using disparities in color components [J].
Li, Haodong ;
Li, Bin ;
Tan, Shunquan ;
Huang, Jiwu .
SIGNAL PROCESSING, 2020, 174
[18]   Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection [J].
Li, Jiaming ;
Xie, Hongtao ;
Li, Jiahong ;
Wang, Zhongyuan ;
Zhang, Yongdong .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :6454-6463
[19]   Face X-ray for More General Face Forgery Detection [J].
Li, Lingzhi ;
Bao, Jianmin ;
Zhang, Ting ;
Yang, Hao ;
Chen, Dong ;
Wen, Fang ;
Guo, Baining .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :5000-5009
[20]   Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics [J].
Li, Yuezun ;
Yang, Xin ;
Sun, Pu ;
Qi, Honggang ;
Lyu, Siwei .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :3204-3213