DIFBFSR: BLIND FACE SUPER-RESOLUTION VIA CONDITIONAL DIFFUSION CONTRACTION

被引:2
作者
Yu, Wei [1 ]
Li, Zonglin [1 ]
Liu, Qinglin [1 ]
Chen, Yufan [1 ]
Zhang, Shengping [1 ]
Lin, Jingbo [2 ]
机构
[1] Harbin Inst Technol, Sch Comp Sci & Technol, Weihai, Peoples R China
[2] Yantai Inst Mat Med, Yantai, Peoples R China
关键词
Blind face super-resolution; diffusion model; face restoration; image generation;
D O I
10.31577/cai_2024_2_369
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Blind Face Super -Resolution (BFSR) has recently gained widespread attention, which aims to super -resolve Low -Resolution (LR) face images with complex unknown degradation to High -Resolution (HR) face images. However, existing BFSR methods suffer from two major limitations. First, most of them are trained on synthetic degradation data pairs with pre -defined degradation models, which leads to poor performance due to the degradation mismatch between other unknown complex degradations in real -world scenarios. Second, some methods rely on hand-crafted face priors as constraints, such as facial landmarks and parsing maps, which require additional callouts and laborious hyperparameter tuning for real cases. To tackle these issues, we propose a simple and effective self -supervised cooperative learning framework via a conditional diffusion contraction method for BFSR, dubbed DifBFSR, which establishes the posterior distribution of HR images from degraded LR images with unknown degradation via a powerful diffusion model without expensive supervised training or additional constraint design. Specifically, we first transform the degraded LR face image to an intermediate HR face prediction with degradation -invariant by a simple Super -Resolution module (SRM), which only relies on self -supervised optimization. To enhance the face pre diction, we propose a Contraction Filter Module (CFM) to gradually contract the restoration error by adaptive dynamic filtering, which efficiently leverages rich na- ture face prior encapsulated in the pre -trained diffusion model through conditional posterior sampling. Finally, by combining the SRM, CFM, and diffusion model in a self -supervised cooperative learning framework, DifBFSR can robustly handle unknown complex degradations, which favorably avoids the cumbersome training and parameter tuning. Extensive qualitative and quantitative experiments on com- plex degraded synthetic and real -world datasets show that our method outperforms state-of-the-art BFSR methods.
引用
收藏
页码:369 / 392
页数:24
相关论文
共 50 条
[11]  
Heusel M, 2017, ADV NEUR IN, V30
[12]  
Ho Jonathan., 2020, ADV NEURAL INFORM PR, V33, P6840
[13]   Analyzing and Improving the Image Quality of StyleGAN [J].
Karras, Tero ;
Laine, Samuli ;
Aittala, Miika ;
Hellsten, Janne ;
Lehtinen, Jaakko ;
Aila, Timo .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), 2020, :8107-8116
[14]  
Karras T, 2018, 6 INT C LEARN REPR I
[15]   A Style-Based Generator Architecture for Generative Adversarial Networks [J].
Karras, Tero ;
Laine, Samuli ;
Aila, Timo .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :4396-4405
[16]  
Kawar B, 2022, ADV NEUR IN
[17]   Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network [J].
Ledig, Christian ;
Theis, Lucas ;
Huszar, Ferenc ;
Caballero, Jose ;
Cunningham, Andrew ;
Acosta, Alejandro ;
Aitken, Andrew ;
Tejani, Alykhan ;
Totz, Johannes ;
Wang, Zehan ;
Shi, Wenzhe .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :105-114
[18]   SRDiff: Single image super-resolution with diffusion probabilistic models [J].
Li, Haoying ;
Yang, Yifan ;
Chang, Meng ;
Chen, Shiqi ;
Feng, Huajun ;
Xu, Zhihai ;
Li, Qi ;
Chen, Yueting .
NEUROCOMPUTING, 2022, 479 :47-59
[19]   Blind Face Restoration via Deep Multi-scale Component Dictionaries [J].
Li, Xiaoming ;
Chen, Chaofeng ;
Zhou, Shangchen ;
Lin, Xianhui ;
Zuo, Wangmeng ;
Zhang, Lei .
COMPUTER VISION - ECCV 2020, PT IX, 2020, 12354 :399-415
[20]   Enhanced Blind Face Restoration with Multi-Exemplar Images and Adaptive Spatial Feature Fusion [J].
Li, Xiaoming ;
Li, Wenyu ;
Ren, Dongwei ;
Zhang, Hongzhi ;
Wang, Meng ;
Zuo, Wangmeng .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :2703-2712