Temporally Consistent Semantic Video Editing

被引:16
作者
Xu, Yiran [1 ]
AlBahar, Badour [2 ]
Huang, Jia-Bin [1 ]
机构
[1] Univ Maryland, College Pk, MD 20742 USA
[2] Virginia Tech, Blacksburg, VA USA
来源
COMPUTER VISION - ECCV 2022, PT XV | 2022年 / 13675卷
关键词
Video editing; GAN editing; Video consistency;
D O I
10.1007/978-3-031-19784-0_21
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Generative adversarial networks (GANs) have demonstrated impressive image generation quality and semantic editing capability of real images, e.g., changing object classes, modifying attributes, or transferring styles. However, applying these GAN-based editing to a video independently for each frame inevitably results in temporal flickering artifacts. We present a simple yet effective method to facilitate temporally coherent video editing. Our core idea is to minimize the temporal photometric inconsistency by optimizing both the latent code and the pre-trained generator. We evaluate the quality of our editing on different domains and GAN inversion techniques and show favorable results against the baselines.
引用
收藏
页码:357 / 374
页数:18
相关论文
共 64 条
[1]  
Abdal R., 2020, P IEEECVF C COMPUTER
[2]   StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows [J].
Abdal, Rameen ;
Zhu, Peihao ;
Mitra, Niloy J. ;
Wonka, Peter .
ACM TRANSACTIONS ON GRAPHICS, 2021, 40 (03)
[3]   Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? [J].
Abdal, Rameen ;
Qin, Yipeng ;
Wonka, Peter .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :4431-4440
[4]   HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms [J].
Afifi, Mahmoud ;
Brubaker, Marcus A. ;
Brown, Michael S. .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :7937-7946
[5]  
Alaluf Y, 2021, Arxiv, DOI [arXiv:2102.02754, DOI 10.48550/ARXIV.2102.02754]
[6]   ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement [J].
Alaluf, Yuval ;
Patashnik, Or ;
Cohen-Or, Daniel .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :6691-6700
[7]  
Alaluf Yuval, 2022, arXiv
[8]  
Bau D, 2020, Arxiv, DOI [arXiv:2005.07727, 10.48550/arXiv:2005.07727]
[9]   Blind Video Temporal Consistency [J].
Bonneel, Nicolas ;
Tompkin, James ;
Sunkavalli, Kalyan ;
Sun, Deqing ;
Paris, Sylvain ;
Pfister, Hanspeter .
ACM TRANSACTIONS ON GRAPHICS, 2015, 34 (06)
[10]   FlyMap: Interacting with Maps Projected from a Drone [J].
Brock, Anke M. ;
Chatain, Julia ;
Park, Michelle ;
Fang, Tommy ;
Hachet, Martin ;
Landay, James A. ;
Cauchard, Jessica R. .
PROCEEDINGS PERVASIVE DISPLAYS 2018: THE 7TH ACM INTERNATIONAL SYMPOSIUM ON PERVASIVE DISPLAYS, 2018,