Navigating the GAN Parameter Space for Semantic Image Editing

被引:39
作者
Cherepkov, Anton [1 ,2 ]
Voynov, Andrey [1 ]
Babenko, Artem [1 ,3 ]
机构
[1] Yandex, Moscow, Russia
[2] Moscow Inst Phys & Technol, Moscow, Russia
[3] HSE Univ, Moscow, Russia
来源
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 | 2021年
关键词
D O I
10.1109/CVPR46437.2021.00367
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Generative Adversarial Networks (GANs) are currently an indispensable tool for visual editing, being a standard component of image-to-image translation and image restoration pipelines. Furthermore, GANs are especially advantageous for controllable generation since their latent spaces contain a wide range of interpretable directions, well suited for semantic editing operations. By gradually changing latent codes along these directions, one can produce impressive visual effects, unattainable without GANs. In this paper, we significantly expand the range of visual effects achievable with the state-of-the-art models, like StyleGAN2. In contrast to existing works, which mostly operate by latent codes, we discover interpretable directions in the space of the generator parameters. By several simple methods, we explore this space and demonstrate that it also contains a plethora of interpretable directions, which are an excellent source of non-trivial semantic manipulations. The discovered manipulations cannot be achieved by transforming the latent codes and can be used to edit both synthetic and real images. We release our code and models and hope they will serve as a handy tool for further efforts on GAN-based image editing.
引用
收藏
页码:3670 / 3679
页数:10
相关论文
共 33 条
[1]  
[Anonymous], 2017, IEEE I CONF COMP VIS, DOI DOI 10.1109/ICCV.2017.244
[2]  
Bau D., 2019, P ICLR
[3]   Semantic Photo Manipulation with a Generative Image Prior [J].
Bau, David ;
Strobelt, Hendrik ;
Peebles, William ;
Wulff, Jonas ;
Zhou, Bolei ;
Zhu, Jun-Yan ;
Torralba, Antonio .
ACM TRANSACTIONS ON GRAPHICS, 2019, 38 (04)
[4]   StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation [J].
Choi, Yunjey ;
Choi, Minje ;
Kim, Munyoung ;
Ha, Jung-Woo ;
Kim, Sunghun ;
Choo, Jaegul .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :8789-8797
[5]   Editing in Style: Uncovering the Local Semantics of GANs [J].
Collins, Edo ;
Bala, Raja ;
Price, Bob ;
Susstrunk, Sabine .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :5770-5779
[6]   GANalyze: Toward Visual Definitions of Cognitive Image Properties [J].
Goetschalckx, Lore ;
Andonian, Alex ;
Oliva, Aude ;
Isola, Phillip .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :5743-5752
[7]  
Goodfellow IJ, 2014, ADV NEUR IN, V27, P2672
[8]   Image Processing Using Multi-Code GAN Prior [J].
Gu, Jinjin ;
Shen, Yujun ;
Zhou, Bolei .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :3009-3018
[9]  
Harkonen E., 2020, P 34 INT C NEUR INF
[10]   Deep Residual Learning for Image Recognition [J].
He, Kaiming ;
Zhang, Xiangyu ;
Ren, Shaoqing ;
Sun, Jian .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :770-778