Uncertainty-aware image inpainting with adaptive feedback network

被引:5
作者
Ma, Xin [1 ]
Zhou, Xiaoqiang [3 ,4 ]
Huang, Huaibo [3 ]
Jia, Gengyun [2 ]
Wang, Yaohui [5 ]
Chen, Xinyuan [5 ]
Chen, Cunjian [1 ]
机构
[1] Monash Univ, Clayton, Australia
[2] Nanjing Univ Posts & Telecommun, Sch Commun & Informat Engn, Nanjing, Peoples R China
[3] Chinese Acad Sci, Inst Automat, Nanjing, Peoples R China
[4] Univ Sci & Technol China, Hefei, Peoples R China
[5] Shanghai Artificial Intelligence Lab, Shanghai, Peoples R China
基金
国家重点研发计划;
关键词
Image inpainting; Uncertainty estimation; Feedback mechanism;
D O I
10.1016/j.eswa.2023.121148
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
While most image inpainting methods perform well on small image defects, they still struggle to deliver satisfactory results on large holes due to insufficient image guidance. To address this challenge, this paper proposes an uncertainty-aware adaptive feedback network (U2AFN), which incorporates an adaptive feedback mechanism to refine inpainting regions progressively. U2AFN predicts both an uncertainty map and an inpainting result simultaneously. During each iteration, the adaptive integration feedback block utilizes inpainting pixels with low uncertainty to guide the subsequent learning iteration. This process leads to a gradual reduction in uncertainty and produces more reliable inpainting outcomes. Our approach is extensively evaluated and compared on multiple datasets, demonstrating its superior performance over existing methods. The code is available at: https://codeocean.com/capsule/1901983/tree.
引用
收藏
页数:8
相关论文
共 48 条
[1]   PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing [J].
Barnes, Connelly ;
Shechtman, Eli ;
Finkelstein, Adam ;
Goldman, Dan B. .
ACM TRANSACTIONS ON GRAPHICS, 2009, 28 (03)
[2]   Simultaneous structure and texture image inpainting [J].
Bertalmio, M ;
Vese, L ;
Sapiro, G ;
Osher, S .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2003, 12 (08) :882-889
[3]   Image inpainting [J].
Bertalmio, M ;
Sapiro, G ;
Caselles, V ;
Ballester, C .
SIGGRAPH 2000 CONFERENCE PROCEEDINGS, 2000, :417-424
[4]   Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving [J].
Choi, Jiwoong ;
Chun, Dayoung ;
Kim, Hyun ;
Lee, Hyuk-Jae .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :502-511
[5]   Image Melding: Combining Inconsistent Images using Patch-based Synthesis [J].
Darabi, Soheil ;
Shechtman, Eli ;
Barnes, Connelly ;
Goldman, Dan B. ;
Sen, Pradeep .
ACM TRANSACTIONS ON GRAPHICS, 2012, 31 (04)
[6]  
Deng J, 2009, PROC CVPR IEEE, P248, DOI 10.1109/CVPRW.2009.5206848
[7]   What Makes Paris Look Like Paris? [J].
Doersch, Carl ;
Singh, Saurabh ;
Gupta, Abhinav ;
Sivic, Josef ;
Efros, Alexei A. .
COMMUNICATIONS OF THE ACM, 2015, 58 (12) :103-110
[8]   Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding [J].
Dong, Qiaole ;
Cao, Chenjie ;
Fu, Yanwei .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, :11348-11358
[9]   Fragment-based image completion [J].
Drori, I ;
Cohen-Or, D ;
Yeshurun, H .
ACM TRANSACTIONS ON GRAPHICS, 2003, 22 (03) :303-312
[10]  
Goodfellow IJ, 2014, ADV NEUR IN, V27, P2672