Coarse-to-Fine Task-Driven Inpainting for Geoscience Images

被引:2
|
作者
Sun, Huiming [1 ]
Ma, Jin [1 ]
Guo, Qing [2 ,3 ]
Zou, Qin [4 ]
Song, Shaoyue [5 ]
Lin, Yuewei [6 ]
Yu, Hongkai [1 ]
机构
[1] Cleveland State Univ, Washkewicz Coll Engn, EECS Dept, Cleveland, OH 44115 USA
[2] ASTAR, Ctr Frontier AI Res CFAR, Singapore 138632, Singapore
[3] ASTAR, Inst High Performance Comp IHPC, Singapore 138632, Singapore
[4] Wuhan Univ, Sch Comp Sci, Wuhan 430072, Peoples R China
[5] Beijing Univ Technol, Fac Informat Technol, Beijing 100021, Peoples R China
[6] Brookhaven Natl Lab, Upton, NY 11973 USA
关键词
Image recognition; Maintenance engineering; Image inpainting; geoscience images; coarse-tofine; task-driven; QUALITY ASSESSMENT; FUSION;
D O I
10.1109/TCSVT.2023.3276719
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
The processing and recognition of geoscience images have wide applications. Most of existing researches focus on understanding the high-quality geoscience images by assuming that all the images are clear. However, in many real-world cases, the geoscience images might contain occlusions during the image acquisition. This problem actually implies the image inpainting problem in computer vision and multimedia. As far as we know, all the existing image inpainting algorithms learn to repair the occluded regions for a better visualization quality, they are excellent for natural images but not good enough for geoscience images, and they never consider the following gescience task when developing inpainting methods. This paper aims to repair the occluded regions for a better geoscience task performance and advanced visualization quality simultaneously, without changing the current deployed deep learning based geoscience models. Because of the complex context of geoscience images, we propose a coarse-to-fine encoder-decoder network with the help of designed coarse-to-fine adversarial context discriminators to reconstruct the occluded image regions. Due to the limited data of geoscience images, we propose a MaskMix based data augmentation method, which augments inpainting masks instead of augmenting original images, to exploit the limited geoscience image data. The experimental results on three public geoscience datasets for remote sensing scene recognition, cross-view geolocation and semantic segmentation tasks respectively show the effectiveness and accuracy of the proposed method. The code is available at: https://github.com/HMS97/Task-driven-Inpainting.
引用
收藏
页码:7170 / 7182
页数:13
相关论文
共 50 条
  • [41] Coarse-to-Fine Nutrition Prediction
    Wang, Binglu
    Bu, Tianci
    Hu, Zaiyi
    Yang, Le
    Zhao, Yongqiang
    Li, Xuelong
    IEEE TRANSACTIONS ON MULTIMEDIA, 2024, 26 (26) : 3651 - 3662
  • [42] ON THE COARSE-TO-FINE STRATEGY IN STEREOMATCHING
    PRAZDNY, K
    BULLETIN OF THE PSYCHONOMIC SOCIETY, 1987, 25 (02) : 92 - 94
  • [43] A Coarse-to-Fine Network for Ship Detection in Optical Remote Sensing Images
    Wu, Yue
    Ma, Wenping
    Gong, Maoguo
    Bai, Zhuangfei
    Zhao, Wei
    Guo, Qiongqiong
    Chen, Xiaobo
    Miao, Qiguang
    REMOTE SENSING, 2020, 12 (02)
  • [44] Coarse-to-Fine Evolutionary Method for Fast Horizon Detection in Maritime Images
    Ganbold, Uuganbayar
    Sato, Junya
    Akashi, Takuya
    IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2021, E104D (12): : 2226 - 2236
  • [45] Exploiting Coarse-to-Fine Task Transfer for Aspect-Level Sentiment Classification
    Li, Zheng
    Wei, Ying
    Zhang, Yu
    Zhang, Xiang
    Li, Xin
    THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FIRST INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / NINTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2019, : 4253 - 4260
  • [46] Coarse-to-Fine Speech Emotion Recognition Based on Multi-Task Learning
    Zhao, Huijuan
    Ye, Ning
    Wang, Ruchuan
    JOURNAL OF SIGNAL PROCESSING SYSTEMS FOR SIGNAL IMAGE AND VIDEO TECHNOLOGY, 2021, 93 (2-3): : 299 - 308
  • [47] Does coarse-to-fine spatial frequency processing depend on object categorization task?
    Craddock, M.
    Martinovic, J.
    Mueller, M.
    PERCEPTION, 2013, 42 : 125 - 125
  • [48] Coarse-to-Fine Speech Emotion Recognition Based on Multi-Task Learning
    Zhao Huijuan
    Ye Ning
    Wang Ruchuan
    Journal of Signal Processing Systems, 2021, 93 : 299 - 308
  • [49] Incremental and Multi-Task Learning Strategies for Coarse-To-Fine Semantic Segmentation
    Mel, Mazen
    Michieli, Umberto
    Zanuttigh, Pietro
    TECHNOLOGIES, 2020, 8 (01)
  • [50] Task-driven Progressive Part Localization for Fine-grained Recognition
    Huang, Chen
    He, Zhihai
    2016 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2016), 2016,