Temporally Consistent Semantic Video Editing

被引:16
作者
Xu, Yiran [1 ]
AlBahar, Badour [2 ]
Huang, Jia-Bin [1 ]
机构
[1] Univ Maryland, College Pk, MD 20742 USA
[2] Virginia Tech, Blacksburg, VA USA
来源
COMPUTER VISION - ECCV 2022, PT XV | 2022年 / 13675卷
关键词
Video editing; GAN editing; Video consistency;
D O I
10.1007/978-3-031-19784-0_21
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Generative adversarial networks (GANs) have demonstrated impressive image generation quality and semantic editing capability of real images, e.g., changing object classes, modifying attributes, or transferring styles. However, applying these GAN-based editing to a video independently for each frame inevitably results in temporal flickering artifacts. We present a simple yet effective method to facilitate temporally coherent video editing. Our core idea is to minimize the temporal photometric inconsistency by optimizing both the latent code and the pre-trained generator. We evaluate the quality of our editing on different domains and GAN inversion techniques and show favorable results against the baselines.
引用
收藏
页码:357 / 374
页数:18
相关论文
共 64 条
[21]   Transforming and Projecting Images into Class-Conditional Generative Networks [J].
Huh, Minyoung ;
Zhang, Richard ;
Zhu, Jun-Yan ;
Paris, Sylvain ;
Hertzmann, Aaron .
COMPUTER VISION - ECCV 2020, PT II, 2020, 12347 :17-34
[22]  
Gulrajani I, 2017, ADV NEUR IN, V30
[23]   StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation [J].
Jang, Wonjong ;
Ju, Gwangjin ;
Jung, Yucheol ;
Yang, Jiaolong ;
Tong, Xin ;
Lee, Seungyong .
ACM TRANSACTIONS ON GRAPHICS, 2021, 40 (04)
[24]   Analyzing and Improving the Image Quality of StyleGAN [J].
Karras, Tero ;
Laine, Samuli ;
Aittala, Miika ;
Hellsten, Janne ;
Lehtinen, Jaakko ;
Aila, Timo .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), 2020, :8107-8116
[25]  
Karras T., 2021, PROCEEDING NEURIPS
[26]  
Karras T, 2020, ADV NEUR IN, V33
[27]   A Style-Based Generator Architecture for Generative Adversarial Networks [J].
Karras, Tero ;
Laine, Samuli ;
Aila, Timo .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :4396-4405
[28]  
Karras Tero, 2017, CoRR
[29]   Layered Neural Atlases for Consistent Video Editing [J].
Kasten, Yoni ;
Ofri, Dolev ;
Wang, Oliver ;
Dekel, Tali .
ACM TRANSACTIONS ON GRAPHICS, 2021, 40 (06)
[30]  
Kingma DP, 2014, ADV NEUR IN, V27