ISRnet: Compressed Image Inpainting Based on Generative Adversarial Network

被引:0
作者
Huang, Junjian [1 ]
Zheng, Mao [1 ]
Li, Zhizhang [1 ]
He, Xing [1 ]
Wen, Shiping [2 ]
机构
[1] Southwest Univ, Coll Elect & Informat Engn, Chongqing Key Lab Nonlinear Circuits & Intelligent, Chongqing 400715, Peoples R China
[2] Univ Technol, Australian Artificial Intelligence Inst AAII, Fac Engn & Informat Technol, Sydney, NSW 2007, Australia
来源
IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE | 2024年
关键词
Image restoration; Superresolution; Image resolution; Generative adversarial networks; Image reconstruction; Generators; Training; super resolution; image inpainting;
D O I
10.1109/TETCI.2024.3446690
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In recent years, significant advancements have been made in the domain of image restoration, particularly in the context of repairing damaged images and super-resolution reconstruction, primarily owing to the emergence of deep learning techniques. However, during the course of transmission across various media devices, the original image quality may deteriorate due to factors such as the network environment, hardware constraints, and related conditions. Moreover, the restoration process becomes increasingly challenging when the original image quality is low or compromised. Presently, prevailing methods involve repairing the image prior to performing super-resolution reconstruction. However, this methodology typically relies on the utilization of multiple autonomous models, where the efficacy and time efficiency are not optimal. In light of this, we propose a novel neural network model based on GAN, termed ISRnet, designed to repair damaged compressed images. Our method is the first to leverage GAN networks specifically for compressed image restoration. ISRnet integrates the principles of image inpainting and super-resolution reconstruction, enabling the transformation of low-resolution images into high-resolution counterparts during the restoration process, thereby achieving superior restoration outcomes. Despite the partial increase in bias, this approach counteracts the variability inherent in a singularly trained neural network model. Compared to a single neural network model trained on the same dataset, our model demonstrates reduced variance and diminished sensitivity to data, thereby achieving optimal restoration quality and expedited repair speeds for damaged compressed images. Consequently, our proposed methodology presents a promising avenue for the utilization of neural networks in repairing damaged compressed images.
引用
收藏
页数:11
相关论文
共 26 条
[1]  
Arjovsky M, 2017, PR MACH LEARN RES, V70
[2]   Region filling and object removal by exemplar-based image inpainting [J].
Criminisi, A ;
Pérez, P ;
Toyama, K .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2004, 13 (09) :1200-1212
[3]   Learning a Deep Convolutional Network for Image Super-Resolution [J].
Dong, Chao ;
Loy, Chen Change ;
He, Kaiming ;
Tang, Xiaoou .
COMPUTER VISION - ECCV 2014, PT IV, 2014, 8692 :184-199
[4]   Fragment-based image completion [J].
Drori, I ;
Cohen-Or, D ;
Yeshurun, H .
ACM TRANSACTIONS ON GRAPHICS, 2003, 22 (03) :303-312
[5]   Advances and challenges in Super-Resolution [J].
Farsiu, S ;
Robinson, D ;
Elad, M ;
Milanfar, P .
INTERNATIONAL JOURNAL OF IMAGING SYSTEMS AND TECHNOLOGY, 2004, 14 (02) :47-57
[6]   Fast and robust multiframe super resolution [J].
Farsiu, S ;
Robinson, MD ;
Elad, M ;
Milanfar, P .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2004, 13 (10) :1327-1344
[7]   Generative Adversarial Networks [J].
Goodfellow, Ian ;
Pouget-Abadie, Jean ;
Mirza, Mehdi ;
Xu, Bing ;
Warde-Farley, David ;
Ozair, Sherjil ;
Courville, Aaron ;
Bengio, Yoshua .
COMMUNICATIONS OF THE ACM, 2020, 63 (11) :139-144
[8]  
Gulrajani I., 2017, Improved training of wasserstein gans, P5769
[9]   Image Inpainting via Conditional Texture and Structure Dual Generation [J].
Guo, Xiefan ;
Yang, Hongyu ;
Huang, Di .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :14114-14123
[10]   Scene Completion Using Millions of Photographs [J].
Hays, James ;
Efros, Alexei A. .
COMMUNICATIONS OF THE ACM, 2008, 51 (10) :87-94