SimSwap: An Efficient Framework For High Fidelity Face Swapping

被引:215
作者
Chen, Renwang [1 ]
Chen, Xuanhong [1 ]
Ni, Bingbing [1 ]
Ge, Yanhao [2 ]
机构
[1] Shanghai Jiao Tong Univ, Shanghai, Peoples R China
[2] Tencent, Shanghai, Peoples R China
来源
MM '20: PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA | 2020年
基金
美国国家科学基金会;
关键词
generative adversarial network; face swapping; image translation;
D O I
10.1145/3394171.3413630
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We propose an efficient framework, called Simple Swap (SimtSwap), aiming for generalized and high fidelity face swapping. In contrast to previous approaches that either lack the ability to generalize to arbitrary identity or fail to preserve attributes like facial expression and gaze direction, our framework is capable of transferring the identity of an arbitrary source face into an arbitrary target face while preserving the attributes of the target face. We overcome the above defects in the following two ways. First, we present the ID Injection Module (IIM) which transfers the identity information of the source face into the target face at feature level. By using this module, we extend the architecture of an identity-specific face swapping algorithm to a framework for arbitrary face swapping. Second, we propose the Weak Feature Matching Loss which efficiently helps our framework to preserve the facial attributes in an implicit way. Extensive experiments on wild faces demonstrate that our SimSwap is able to achieve competitive identity performance while preserving attributes better than previous state-of-the-art methods.
引用
收藏
页码:2003 / 2011
页数:9
相关论文
共 34 条
[1]  
[Anonymous], 2017, P INT C LEARN REPR
[2]   Towards Open-Set Identity Preserving Face Synthesis [J].
Bao, Jianmin ;
Chen, Dong ;
Wen, Fang ;
Li, Houqiang ;
Hua, Gang .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :6713-6722
[3]   Face swapping: Automatically replacing faces in photographs [J].
Bitouk, Dmitri ;
Kumar, Neeraj ;
Dhillon, Samreen ;
Belhumeur, Peter ;
Nayar, Shree K. .
ACM TRANSACTIONS ON GRAPHICS, 2008, 27 (03)
[4]   Exchanging faces in images [J].
Blanz, V ;
Scherbaum, K ;
Vetter, T ;
Seidel, HP .
COMPUTER GRAPHICS FORUM, 2004, 23 (03) :669-676
[5]  
Brock Andrew, 2019, INT C LEARNING REPRE
[6]   VGGFace2: A dataset for recognising faces across pose and age [J].
Cao, Qiong ;
Shen, Li ;
Xie, Weidi ;
Parkhi, Omkar M. ;
Zisserman, Andrew .
PROCEEDINGS 2018 13TH IEEE INTERNATIONAL CONFERENCE ON AUTOMATIC FACE & GESTURE RECOGNITION (FG 2018), 2018, :67-74
[7]  
DeepFakes, 2020, FACESWAP
[8]   ArcFace: Additive Angular Margin Loss for Deep Face Recognition [J].
Deng, Jiankang ;
Guo, Jia ;
Xue, Niannan ;
Zafeiriou, Stefanos .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :4685-4694
[9]  
foamliu, 2019, INSIGHTFACE V2
[10]  
Goodfellow IJ, 2014, ADV NEUR IN, V27, P2672