Progressive Semantic Reasoning for Image Inpainting

被引:2
作者
Jin, Junjie [1 ]
Hu, Xinrong [1 ]
He, Kai [1 ]
Peng, Tao [1 ]
Liu, Junping [1 ]
Yang, Jie [2 ]
机构
[1] Wuhan Text Univ, Sch Math & Comp Sci, Engn Res Ctr Hubei Prov Clothing Informat, Wuhan, Hubei, Peoples R China
[2] Univ Wollongong, Sch Comp & Informat Technol, Wollongong, NSW, Australia
来源
WEB CONFERENCE 2021: COMPANION OF THE WORLD WIDE WEB CONFERENCE (WWW 2021) | 2021年
关键词
image inpainting; semantic reasoning; feature reconstruction; attention mechanism;
D O I
10.1145/3442442.3451142
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Image inpainting aims to reconstruct the missing or unknown region for a given image. As one of the most important topics from image processing, this task has attracted increasing research interest over the past few decades. Learning-based methods have been employed to solve this task, and achieved superior performance. Nevertheless, existing methods often produce artificial traces, due to the lack of constraints on image characterization under different semantics. To accommodate this issue, we propose a novel artistic Progressive Semantic Reasoning (PSR) network in this paper, which is composed of three shared parameters from the generation network superposition. More precisely, the proposed PSR algorithm follows a typical end-to-end training procedure, that learns low-level semantic features and further transfers them to a high-level semantic network for inpainting purposes. Furthermore, a simple but effective Cross Feature Reconstruction (CFR) strategy is proposed to tradeoff semantic information from different levels. Empirically, the proposed approach is evaluated via intensive experiments using a variety of real-world datasets. The results confirm the effectiveness of our algorithm compared with other state-of-the-art methods. The source code can be found from https://github.com/sfwyly/PSR-Net.
引用
收藏
页码:68 / 76
页数:9
相关论文
共 30 条
[1]   PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing [J].
Barnes, Connelly ;
Shechtman, Eli ;
Finkelstein, Adam ;
Goldman, Dan B. .
ACM TRANSACTIONS ON GRAPHICS, 2009, 28 (03)
[2]   Region filling and object removal by exemplar-based image inpainting [J].
Criminisi, A ;
Pérez, P ;
Toyama, K .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2004, 13 (09) :1200-1212
[3]   Image Inpainting Using Nonlocal Texture Matching and Nonlinear Filtering [J].
Ding, Ding ;
Ram, Sundaresh ;
Rodriguez, Jeffrey J. .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (04) :1705-1719
[4]   What Makes Paris Look like Paris? [J].
Doersch, Carl ;
Singh, Saurabh ;
Gupta, Abhinav ;
Sivic, Josef ;
Efros, Alexei A. .
ACM TRANSACTIONS ON GRAPHICS, 2012, 31 (04)
[5]   Generative Adversarial Networks [J].
Goodfellow, Ian ;
Pouget-Abadie, Jean ;
Mirza, Mehdi ;
Xu, Bing ;
Warde-Farley, David ;
Ozair, Sherjil ;
Courville, Aaron ;
Bengio, Yoshua .
COMMUNICATIONS OF THE ACM, 2020, 63 (11) :139-144
[6]  
Hu J, 2018, PROC CVPR IEEE, P7132, DOI [10.1109/TPAMI.2019.2913372, 10.1109/CVPR.2018.00745]
[7]   Globally and Locally Consistent Image Completion [J].
Iizuka, Satoshi ;
Simo-Serra, Edgar ;
Ishikawa, Hiroshi .
ACM TRANSACTIONS ON GRAPHICS, 2017, 36 (04)
[8]  
King DB, 2015, ACS SYM SER, V1214, P1, DOI 10.1021/bk-2015-1214.ch001
[9]   Recurrent Feature Reasoning for Image Inpainting [J].
Li, Jingyuan ;
Wang, Ning ;
Zhang, Lefei ;
Du, Bo ;
Tao, Dacheng .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), 2020, :7757-7765
[10]   Progressive Reconstruction of Visual Structure for Image Inpainting [J].
Li, Jingyuan ;
He, Fengxiang ;
Zhang, Lefei ;
Du, Bo ;
Tao, Dacheng .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :5961-5970