DIFBFSR: BLIND FACE SUPER-RESOLUTION VIA CONDITIONAL DIFFUSION CONTRACTION

被引:2
作者
Yu, Wei [1 ]
Li, Zonglin [1 ]
Liu, Qinglin [1 ]
Chen, Yufan [1 ]
Zhang, Shengping [1 ]
Lin, Jingbo [2 ]
机构
[1] Harbin Inst Technol, Sch Comp Sci & Technol, Weihai, Peoples R China
[2] Yantai Inst Mat Med, Yantai, Peoples R China
关键词
Blind face super-resolution; diffusion model; face restoration; image generation;
D O I
10.31577/cai_2024_2_369
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Blind Face Super -Resolution (BFSR) has recently gained widespread attention, which aims to super -resolve Low -Resolution (LR) face images with complex unknown degradation to High -Resolution (HR) face images. However, existing BFSR methods suffer from two major limitations. First, most of them are trained on synthetic degradation data pairs with pre -defined degradation models, which leads to poor performance due to the degradation mismatch between other unknown complex degradations in real -world scenarios. Second, some methods rely on hand-crafted face priors as constraints, such as facial landmarks and parsing maps, which require additional callouts and laborious hyperparameter tuning for real cases. To tackle these issues, we propose a simple and effective self -supervised cooperative learning framework via a conditional diffusion contraction method for BFSR, dubbed DifBFSR, which establishes the posterior distribution of HR images from degraded LR images with unknown degradation via a powerful diffusion model without expensive supervised training or additional constraint design. Specifically, we first transform the degraded LR face image to an intermediate HR face prediction with degradation -invariant by a simple Super -Resolution module (SRM), which only relies on self -supervised optimization. To enhance the face pre diction, we propose a Contraction Filter Module (CFM) to gradually contract the restoration error by adaptive dynamic filtering, which efficiently leverages rich na- ture face prior encapsulated in the pre -trained diffusion model through conditional posterior sampling. Finally, by combining the SRM, CFM, and diffusion model in a self -supervised cooperative learning framework, DifBFSR can robustly handle unknown complex degradations, which favorably avoids the cumbersome training and parameter tuning. Extensive qualitative and quantitative experiments on com- plex degraded synthetic and real -world datasets show that our method outperforms state-of-the-art BFSR methods.
引用
收藏
页码:369 / 392
页数:24
相关论文
共 50 条
[1]   GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution [J].
Chan, Kelvin C. K. ;
Wang, Xintao ;
Xu, Xiangyu ;
Gu, Jinwei ;
Loy, Chen Change .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :14240-14249
[2]   Super-resolution through neighbor embedding [J].
Chang, H ;
Yeung, DY ;
Xiong, Y .
PROCEEDINGS OF THE 2004 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOL 1, 2004, :275-282
[3]   Progressive Semantic-Aware Style Transformation for Blind Face Restoration [J].
Chen, Chaofeng ;
Li, Xiaoming ;
Yang, Lingbo ;
Lin, Xianhui ;
Zhang, Lei ;
Wong, Kwan-Yee K. .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :11891-11900
[4]   FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors [J].
Chen, Yu ;
Tai, Ying ;
Liu, Xiaoming ;
Shen, Chunhua ;
Yang, Jian .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :2492-2501
[5]   Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration [J].
Chen, Yunjin ;
Pock, Thomas .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2017, 39 (06) :1256-1272
[6]   ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models [J].
Choi, Jooyoung ;
Kim, Sungwon ;
Jeong, Yonghyun ;
Gwon, Youngjune ;
Yoon, Sungroh .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :14347-14356
[7]  
Dhariwal P, 2021, ADV NEUR IN, V34
[8]   CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution [J].
Gao, Guangwei ;
Xu, Zixiang ;
Li, Juncheng ;
Yang, Jian ;
Zeng, Tieyong ;
Qi, Guo-Jun .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2023, 32 :1978-1991
[9]   Generative Adversarial Networks [J].
Goodfellow, Ian ;
Pouget-Abadie, Jean ;
Mirza, Mehdi ;
Xu, Bing ;
Warde-Farley, David ;
Ozair, Sherjil ;
Courville, Aaron ;
Bengio, Yoshua .
COMMUNICATIONS OF THE ACM, 2020, 63 (11) :139-144
[10]   VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder [J].
Gu, Yuchao ;
Wang, Xintao ;
Xie, Liangbin ;
Dong, Chao ;
Li, Gen ;
Shan, Ying ;
Cheng, Ming-Ming .
COMPUTER VISION - ECCV 2022, PT XVIII, 2022, 13678 :126-143