Efficient deep neural network for photo-realistic image super-resolution

被引:24
作者
Ahn, Namhyuk [1 ,2 ]
Kang, Byungkon [3 ]
Sohn, Kyung-Ah [2 ]
机构
[1] NAVER WEBTOON AI, Seoul, South Korea
[2] Ajou Univ, Dept Artificial Intelligence, Suwon, South Korea
[3] SUNY, Dept Comp Sci, Incheon, South Korea
关键词
Super-resolution; Photo-realistic; Convolutional neural network; Efficient model; Adversarial learning; Multi-scale approach;
D O I
10.1016/j.patcog.2022.108649
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Recent progress in deep learning-based models has improved photo-realistic (or perceptual) single-image super-resolution significantly. However, despite their powerful performance, many methods are difficult to apply to real-world applications because of the heavy computational requirements. To facilitate the use of a deep model under such demands, we focus on keeping the network efficient while maintaining its performance. In detail, we design an architecture that implements a cascading mechanism on a residual network to boost the performance with limited resources via multi-level feature fusion. In addition, our proposed model adopts group convolution and recursive schemes in order to achieve extreme efficiency. We further improve the perceptual quality of the output by employing the adversarial learning paradigm and a multi-scale discriminator approach. The performance of our method is investigated through extensive internal experiments and benchmarks using various datasets. Our results show that our models outperform the recent methods with similar complexity, for both traditional pixel-based and perception-based tasks. (C) 2022 The Authors. Published by Elsevier Ltd.
引用
收藏
页数:13
相关论文
共 51 条
[1]   NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study [J].
Agustsson, Eirikur ;
Timofte, Radu .
2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2017, :1122-1131
[2]   Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network [J].
Ahn, Namhyuk ;
Kang, Byungkon ;
Sohn, Kyung-Ah .
COMPUTER VISION - ECCV 2018, PT X, 2018, 11214 :256-272
[3]  
Ahn Namhyuk, 2018, CVPR, P791
[4]   Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding [J].
Bevilacqua, Marco ;
Roumy, Aline ;
Guillemot, Christine ;
Morel, Marie-Line Alberi .
PROCEEDINGS OF THE BRITISH MACHINE VISION CONFERENCE 2012, 2012,
[5]   The 2018 PIRM Challenge on Perceptual Image Super-Resolution [J].
Blau, Yochai ;
Mechrez, Roey ;
Timofte, Radu ;
Michaeli, Tomer ;
Zelnik-Manor, Lihi .
COMPUTER VISION - ECCV 2018 WORKSHOPS, PT V, 2019, 11133 :334-355
[6]   The Perception-Distortion Tradeoff [J].
Blau, Yochai ;
Michaeli, Tomer .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :6228-6237
[7]   GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution [J].
Chan, Kelvin C. K. ;
Wang, Xintao ;
Xu, Xiangyu ;
Gu, Jinwei ;
Loy, Chen Change .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :14240-14249
[8]   Learning a Deep Convolutional Network for Image Super-Resolution [J].
Dong, Chao ;
Loy, Chen Change ;
He, Kaiming ;
Tang, Xiaoou .
COMPUTER VISION - ECCV 2014, PT IV, 2014, 8692 :184-199
[9]  
Gatys LA, 2015, ADV NEUR IN, V28
[10]  
Glorot X., 2010, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, P249, DOI DOI 10.1109/LGRS.2016.2565705