Image-to-Image Translation With Disentangled Latent Vectors for Face Editing

被引:15
|
作者
Dalva, Yusuf [1 ]
Pehlivan, Hamza [2 ]
Hatipoglu, Oyku Irmak [3 ]
Moran, Cansu [4 ]
Dundar, Aysegul [5 ]
机构
[1] Virginia Tech, Dept Comp Sci, Blacksburg, VA 24061 USA
[2] Max Planck Inst, D-80539 Munich, Germany
[3] Swiss Fed Inst Technol, CH-8092 Zurich, Switzerland
[4] Tech Univ Munich, D-80333 Munich, Germany
[5] Bilkent Univ, Dept Comp Sci, TR-06800 Bilkent, Turkiye
关键词
Image translation; generative adversarial net works; latent space manipulation; face attribute editing;
D O I
10.1109/TPAMI.2023.3308102
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We propose an image-to-image translation framework for facial attribute editing with disentangled interpretable latent directions. Facial attribute editing task faces the challenges of targeted attribute editing with controllable strength and disentanglement in the representations of attributes to preserve the other attributes during edits. For this goal, inspired by the latent space factorization works of fixed pretrained GANs, we design the attribute editing by latent space factorization, and for each attribute, we learn a linear direction that is orthogonal to the others. We train these directions with orthogonality constraints and disentanglement losses. To project images to semantically organized latent spaces, we set an encoder-decoder architecture with attention-based skip connections. We extensively compare with previous image translation algorithms and editing with pretrained GAN works. Our extensive experiments show that our method significantly improves over the state-of-the-arts.
引用
收藏
页码:14777 / 14788
页数:12
相关论文
共 50 条
  • [31] Facial Feature Based Image-to-Image Translation Method
    Kang, Shinjin
    KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS, 2020, 14 (12): : 4835 - 4848
  • [32] Image-to-Image Translation for Simplified MRI Muscle Segmentation
    Gadermayr, Michael
    Heckmann, Lotte
    Li, Kexin
    Baehr, Friederike
    Mueller, Madlaine
    Truhn, Daniel
    Merhof, Dorit
    Gess, Burkhard
    FRONTIERS IN RADIOLOGY, 2021, 1
  • [33] Implicit pairs for boosting unpaired image-to-image translation
    Ginger, Yiftach
    Danon, Dov
    Averbuch-Elor, Hadar
    Cohen-Or, Daniel
    VISUAL INFORMATICS, 2020, 4 (04): : 50 - 58
  • [34] Image Generation and Translation with Disentangled Representations
    Hinz, Tobias
    Wermter, Stefan
    2018 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2018,
  • [35] CoPrGAN: Image-to-Image Translation via Content Preservation
    Yu, Xiaoming
    Zhou, Gan
    ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2022, PT III, 2022, 13531 : 37 - 49
  • [36] Complementary, Heterogeneous and Adversarial Networks for Image-to-Image Translation
    Gao, Fei
    Xu, Xingxin
    Yu, Jun
    Shang, Meimei
    Li, Xiang
    Tao, Dacheng
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 : 3487 - 3498
  • [37] UMGAN: Underwater Image Enhancement Network for Unpaired Image-to-Image Translation
    Sun, Boyang
    Mei, Yupeng
    Yan, Ni
    Chen, Yingyi
    JOURNAL OF MARINE SCIENCE AND ENGINEERING, 2023, 11 (02)
  • [38] Unsupervised Image-to-Image Translation with Self-Attention Networks
    Kang, Taewon
    Lee, Kwang Hee
    2020 IEEE INTERNATIONAL CONFERENCE ON BIG DATA AND SMART COMPUTING (BIGCOMP 2020), 2020, : 102 - 108
  • [39] Unified Generative Adversarial Networks for Controllable Image-to-Image Translation
    Tang, Hao
    Liu, Hong
    Sebe, Nicu
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2020, 29 : 8916 - 8929
  • [40] Spatial-Intensity Transforms for Medical Image-to-Image Translation
    Wang, Clinton J.
    Rost, Natalia S.
    Golland, Polina
    IEEE TRANSACTIONS ON MEDICAL IMAGING, 2023, 42 (11) : 3362 - 3373