Image inpainting based on deep learning: A review

被引:80
作者
Zhang, Xiaobo [1 ,2 ,3 ]
Zhai, Donghai [1 ]
Li, Tianrui [1 ,2 ,3 ]
Zhou, Yuxin [1 ]
Lin, Yang [1 ]
机构
[1] SouthWest JiaoTong Univ, Sch Comp & Artificial Intelligence, Chengdu 611756, Peoples R China
[2] SouthWest JiaoTong Univ, Artificial Intelligence Res Inst, Chengdu 611756, Peoples R China
[3] SouthWest JiaoTong Univ, Natl Engn Lab Integrated Transportat Big Data Appl, Chengdu 611756, Peoples R China
关键词
Image inpainting; Fusion; Deep learning; CNN; GAN; NEURAL-NETWORKS; OBJECT REMOVAL; COMPLETION; FRAMEWORK; REPRESENTATIONS;
D O I
10.1016/j.inffus.2022.08.033
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Image inpainting is an important research direction in the study of computer vision, and is widely used in image editing and photo inpainting etc. Traditional image inpainting algorithms are often difficult to deal with large-scale image deletion, since these algorithms are prone to inconsistent image semantics. With the rapid development of deep learning (DL) in recent years, the advantages of DL in image processing have become increasingly prominent, it can solve the problems existing in traditional image inpainting algorithms to a certain extent. At present, image inpainting based on deep learning becomes a research hotspot in computer vision. In this article, we systematically summarize and analyze the literature on image inpainting based on deep learning. First, we review the specific research status of deep learning technology in the field of image inpainting in the past 15 years; then, We deeply study and analyze the existing image restoration methods based on different neural network structures and their information fusion methods. In addition, we also classify and summarize the different tasks of image inpainting according to the application scenarios of image inpainting. Finally, we point out some problems that urgently need to be solved for deep learning in the field of image inpainting, provide constructive suggestions and discuss the future development direction.
引用
收藏
页码:74 / 94
页数:21
相关论文
共 175 条
[1]   Removal of occluding construction workers in job site image data using U-Net based context encoders [J].
Angah, Ohay ;
Chen, Albert Y. .
AUTOMATION IN CONSTRUCTION, 2020, 119
[2]  
[Anonymous], 2015, 2015 8 INT C ADV PAT
[3]  
Arjovsky M, 2017, Arxiv, DOI [arXiv:1701.07875, DOI 10.48550/ARXIV.1701.07875]
[4]  
Arjovsky Martin., 2017, arXiv
[5]  
Armanious Karim, 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Proceedings, P3267, DOI 10.1109/ICASSP.2019.8682677
[6]   SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation [J].
Badrinarayanan, Vijay ;
Kendall, Alex ;
Cipolla, Roberto .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2017, 39 (12) :2481-2495
[7]   Filling-in by joint interpolation of vector fields and gray levels [J].
Ballester, C ;
Bertalmio, M ;
Caselles, V ;
Sapiro, G ;
Verdera, J .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2001, 10 (08) :1200-1211
[8]   Simultaneous structure and texture image inpainting [J].
Bertalmio, M ;
Vese, L ;
Sapiro, G ;
Osher, S .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2003, 12 (08) :882-889
[9]   Image inpainting [J].
Bertalmio, M ;
Sapiro, G ;
Caselles, V ;
Ballester, C .
SIGGRAPH 2000 CONFERENCE PROCEEDINGS, 2000, :417-424
[10]   A Comprehensive Framework for Image Inpainting [J].
Bugeau, Aurelie ;
Bertalmio, Marcelo ;
Caselles, Vicent ;
Sapiro, Guillermo .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2010, 19 (10) :2634-2645