Coarse-to-Fine Task-Driven Inpainting for Geoscience Images

被引:2
|
作者
Sun, Huiming [1 ]
Ma, Jin [1 ]
Guo, Qing [2 ,3 ]
Zou, Qin [4 ]
Song, Shaoyue [5 ]
Lin, Yuewei [6 ]
Yu, Hongkai [1 ]
机构
[1] Cleveland State Univ, Washkewicz Coll Engn, EECS Dept, Cleveland, OH 44115 USA
[2] ASTAR, Ctr Frontier AI Res CFAR, Singapore 138632, Singapore
[3] ASTAR, Inst High Performance Comp IHPC, Singapore 138632, Singapore
[4] Wuhan Univ, Sch Comp Sci, Wuhan 430072, Peoples R China
[5] Beijing Univ Technol, Fac Informat Technol, Beijing 100021, Peoples R China
[6] Brookhaven Natl Lab, Upton, NY 11973 USA
关键词
Image recognition; Maintenance engineering; Image inpainting; geoscience images; coarse-tofine; task-driven; QUALITY ASSESSMENT; FUSION;
D O I
10.1109/TCSVT.2023.3276719
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
The processing and recognition of geoscience images have wide applications. Most of existing researches focus on understanding the high-quality geoscience images by assuming that all the images are clear. However, in many real-world cases, the geoscience images might contain occlusions during the image acquisition. This problem actually implies the image inpainting problem in computer vision and multimedia. As far as we know, all the existing image inpainting algorithms learn to repair the occluded regions for a better visualization quality, they are excellent for natural images but not good enough for geoscience images, and they never consider the following gescience task when developing inpainting methods. This paper aims to repair the occluded regions for a better geoscience task performance and advanced visualization quality simultaneously, without changing the current deployed deep learning based geoscience models. Because of the complex context of geoscience images, we propose a coarse-to-fine encoder-decoder network with the help of designed coarse-to-fine adversarial context discriminators to reconstruct the occluded image regions. Due to the limited data of geoscience images, we propose a MaskMix based data augmentation method, which augments inpainting masks instead of augmenting original images, to exploit the limited geoscience image data. The experimental results on three public geoscience datasets for remote sensing scene recognition, cross-view geolocation and semantic segmentation tasks respectively show the effectiveness and accuracy of the proposed method. The code is available at: https://github.com/HMS97/Task-driven-Inpainting.
引用
收藏
页码:7170 / 7182
页数:13
相关论文
共 50 条
  • [21] Learning-driven Coarse-to-Fine Articulated Robot Tracking
    Rauch, Christian
    Ivan, Vladimir
    Hospedales, Timothy
    Shotton, Jamie
    Fallon, Maurice
    2019 INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2019, : 6604 - 6610
  • [22] Learning hyperspectral images from RGB images via a coarse-to-fine CNN
    Mei, Shaohui
    Geng, Yunhao
    Hou, Junhui
    Du, Qian
    SCIENCE CHINA-INFORMATION SCIENCES, 2022, 65 (05)
  • [23] Task-Driven Compression for Collision Encoding Based on Depth Images
    Kulkarni, Mihir
    Alexis, Kostas
    ADVANCES IN VISUAL COMPUTING, ISVC 2023, PT II, 2023, 14362 : 259 - 273
  • [24] 'Coarse-to-fine' cyclopean processing
    Popple, AV
    Findlay, JM
    PERCEPTION, 1999, 28 (02) : 155 - 165
  • [25] Coarse-to-fine face detection
    Fleuret, F
    Geman, D
    INTERNATIONAL JOURNAL OF COMPUTER VISION, 2001, 41 (1-2) : 85 - 107
  • [26] Coarse-to-fine manifold learning
    Castro, R
    Willett, R
    Nowak, R
    2004 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL III, PROCEEDINGS: IMAGE AND MULTIDIMENSIONAL SIGNAL PROCESSING SPECIAL SESSIONS, 2004, : 992 - 995
  • [27] Optic disc detection in retinal images via a coarse-to-fine scheme
    Li, Q. (kenneth_lee_qin@qq.com), 1600, Binary Information Press (10):
  • [28] An unsupervised coarse-to-fine algorithm for blood vessel segmentation in fundus images
    Neto, Luiz Camara
    Ramalho, Geraldo L. B.
    Rocha Neto, Jeova F. S.
    Veras, Rodrigo M. S.
    Medeiros, Fatima N. S.
    EXPERT SYSTEMS WITH APPLICATIONS, 2017, 78 : 182 - 192
  • [29] Coarse-to-Fine Grained Classification
    Huo, Yuqi
    Lu, Yao
    Niu, Yulei
    Lu, Zhiwu
    Wen, Ji-Rong
    PROCEEDINGS OF THE 42ND INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL (SIGIR '19), 2019, : 1033 - 1036
  • [30] Coarse-to-fine dynamic programming
    Raphael, C
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2001, 23 (12) : 1379 - 1390