StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows

被引:262
作者
Abdal, Rameen [1 ]
Zhu, Peihao [1 ]
Mitra, Niloy J. [2 ,3 ]
Wonka, Peter [1 ]
机构
[1] KAUST, Thuwal, Saudi Arabia
[2] UCL, London, England
[3] Adobe Res, San Jose, CA USA
来源
ACM TRANSACTIONS ON GRAPHICS | 2021年 / 40卷 / 03期
关键词
Generative adversarial networks; image editing;
D O I
10.1145/3447648
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
High-quality, diverse, and photorealistic images can now be generated by unconditional GANs (e.g., StyleGAN). However, limited options exist to control the generation process using (semantic) attributes while still preserving the quality of the output. Further, due to the entangled nature of the GAN latent space, performing edits along one attribute can easily result in unwanted changes along other attributes. In this article, in the context of conditional exploration of entangled latent spaces, we investigate the two sub-problems of attribute-conditioned sampling and attribute-controlled editing. We present StyleFlow as a simple, effective, and robust solution to both the sub-problems by formulating conditional exploration as an instance of conditional continuous normalizing flows in the GAN latent space conditioned by attribute features. We evaluate our method using the face and the car latent space of StyleGAN, and demonstrate fine-grained disentangled edits along various attributes on both real photographs and StyleGAN generated images. For example, for faces, we vary camera pose, illumination variation, expression, facial hair, gender, and age. Finally, via extensive qualitative and quantitative comparisons, we demonstrate the superiority of StyleFlow over prior and several concurrent works.
引用
收藏
页数:21
相关论文
共 68 条
  • [21] Hedman P, 2018, SIGGRAPH ASIA'18: SIGGRAPH ASIA 2018 TECHNICAL PAPERS, DOI 10.1145/3272127.3275084
  • [22] Image-to-Image Translation with Conditional Adversarial Networks
    Isola, Phillip
    Zhu, Jun-Yan
    Zhou, Tinghui
    Efros, Alexei A.
    [J]. 30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, : 5967 - 5976
  • [23] Jiang Wentao, 2019, ARXIVCSCV190906956
  • [24] Jiapeng Zhu, 2020, Computer Vision - ECCV 2020. 16th European Conference. Proceedings. Lecture Notes in Computer Science (LNCS 12362), P592, DOI 10.1007/978-3-030-58520-4_35
  • [25] SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color
    Jo, Youngjoo
    Park, Jongyoul
    [J]. 2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 1745 - 1753
  • [26] Karras T., 2019, CoRR
  • [27] A Style-Based Generator Architecture for Generative Adversarial Networks
    Karras, Tero
    Laine, Samuli
    Aila, Timo
    [J]. 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 4396 - 4405
  • [28] Deep Video Portraits
    Kim, Hyeongwoo
    Garrido, Pablo
    Tewari, Ayush
    Xu, Weipeng
    Thies, Justus
    Niessner, Matthias
    Perez, Patrick
    Richardt, Christian
    Zollhofer, Michael
    Theobalt, Christian
    [J]. ACM TRANSACTIONS ON GRAPHICS, 2018, 37 (04):
  • [29] Kim T, 2017, PR MACH LEARN RES, V70
  • [30] Kingma D. P., 2013, ARXIV13126114