SwapInpaint2: Towards high structural consistency in identity-guided inpainting via background-preserving GAN inversion

被引:0
作者
Li, Honglei [1 ,2 ]
Zhang, Yifan [1 ]
Wang, Wenmin [1 ]
Zhang, Shenyong [1 ]
Zhang, Shixiong [1 ]
机构
[1] Macau Univ Sci & Technol, Sch Comp Sci & Engn, Taipa, Macao, Peoples R China
[2] Chongqing Polytech Univ Elect Technol, Chongqing Key Lab Big Data Intelligence & Privacy, Chongqing, Peoples R China
关键词
Identity-guided face inpainting; GAN inversion; Image synthesis;
D O I
10.1016/j.patcog.2024.110969
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this work, we propose SwapInpaint2 to enhance the naturalness of identity-guided face inpainting. The previous version, SwapInpaint, relied on a generic inpainting model for content infer, bringing the problems of unnatural structure to the results. Our method addresses this issue by utilizing an inversion-based Background- preserving Attribute Extractor and an improved Embedding Integration Generator to provide high-quality attribute embeddings and produce high-naturalistic results. Comparison experiments with state-of-the-art models demonstrate that our new method achieves superior structural consistency and stronger style alignment both in terms of quality and quantity.
引用
收藏
页数:11
相关论文
共 40 条
  • [1] A model validation approach to texture recognition and inpainting
    Al-Takrouri, Saleh
    Savkin, Andrey V.
    [J]. PATTERN RECOGNITION, 2010, 43 (06) : 2054 - 2067
  • [2] Facial Structure Guided GAN for Identity-preserved Face Image De-occlusion
    Cheung, Yiu-ming
    Li, Mengke
    Zou, Rong
    [J]. PROCEEDINGS OF THE 2021 INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL (ICMR '21), 2021, : 46 - 54
  • [3] Deng Q., 2024, Pattern Recognit.
  • [4] Eye In-Painting with Exemplar Generative Adversarial Networks
    Dolhansky, Brian
    Ferrer, Cristian Canton
    [J]. 2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, : 7902 - 7911
  • [5] Ge S., 2024, INT C MACH LEARN ICM
  • [6] Occluded Face Recognition in the Wild by Identity-Diversity Inpainting
    Ge, Shiming
    Li, Chenyu
    Zhao, Shengwei
    Zeng, Dan
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2020, 30 (10) : 3387 - 3397
  • [7] Goodfellow IJ, 2014, ADV NEUR IN, V27, P2672
  • [8] E2F-Net: Eyes-to-face inpainting via StyleGAN latent space
    Hassanpour, Ahmad
    Jamalbafrani, Fatemeh
    Yang, Bian
    Raja, Kiran
    Veldhuis, Raymond
    Fierrez, Julian
    [J]. PATTERN RECOGNITION, 2024, 152
  • [9] Huang GB, 2008, LABELED FACES WILD D
  • [10] Karras Tero, 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings, P8107, DOI 10.1109/CVPR42600.2020.00813