From degrade to upgrade: Learning a self-supervised degradation guided adaptive network for blind remote sensing image super-resolution

被引:174
作者
Xiao, Yi [1 ]
Yuan, Qiangqiang [1 ]
Jiang, Kui [2 ]
He, Jiang [1 ]
Wang, Yuan [1 ]
Zhang, Liangpei [3 ]
机构
[1] Wuhan Univ, Sch Geodesy & Geomat, Wuhan, Hubei, Peoples R China
[2] Huawei Technol, Cloud BU, Hangzhou, Zhejiang, Peoples R China
[3] Wuhan Univ, State Key Lab Informat Engn Surveying Mapping & Re, Wuhan, Hubei, Peoples R China
基金
中国国家自然科学基金;
关键词
Blind super-resolution; Self-supervised; Contrastive learning; Remote sensing image; Deep learning; RESOLUTION; FUSION;
D O I
10.1016/j.inffus.2023.03.021
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Over the past few years, single image super-resolution (SR) has become a hotspot in the remote sensing area, and numerous methods have made remarkable progress in this fundamental task. However, they usually rely on the assumption that images suffer from a fixed known degradation process, e.g., bicubic downsampling. To save us from performance drop when real-world distribution deviates from the naive assumption, blind image super-resolution for multiple and unknown degradations has been explored. Nevertheless, the lack of a real-world dataset and the challenge of reasonable degradation estimation hinder us from moving forward. In this paper, a self-supervised degradation-guided adaptive network is proposed to mitigate the domain gap between simulation and reality. Firstly, the complicated degradations are characterized by robust representations in embedding space, which promote adaptability to the downstream SR network with degradation priors. Specifically, we incorporated contrastive learning to blind remote sensing image SR, which guides the reconstruction process by encouraging the positive representations (relevant information) while punishing the negatives. Besides, an effective dual-wise feature modulation network is proposed for feature adaptation. With the guide of degradation representations, we conduct modulation on feature and channel dimensions to transform the low-resolution features into the desired domain that is suitable for reconstructing high-resolution images. Extensive experiments on three mainstream datasets have demonstrated our superiority against state-of-the-art methods. Our source code can be found at https://github.com/XY-boy/DRSR
引用
收藏
页码:297 / 311
页数:15
相关论文
共 81 条
[1]   Densely Residual Laplacian Super-Resolution [J].
Anwar, Saeed ;
Barnes, Nick .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (03) :1192-1204
[2]   A novel image decomposition-based hybrid technique with super-resolution method for multi-focus image fusion [J].
Aymaz, Samet ;
Kose, Cemal .
INFORMATION FUSION, 2019, 45 :113-127
[3]   MLCA2F: Multi-Level Context Attentional Feature Fusion for COVID-19 lesion segmentation from CT scans [J].
Bakkouri, Ibtissam ;
Afdel, Karim .
SIGNAL IMAGE AND VIDEO PROCESSING, 2023, 17 (04) :1181-1188
[4]   Computer-aided diagnosis (CAD) system based on multi-layer feature fusion network for skin lesion recognition in dermoscopy images [J].
Bakkouri, Ibtissam ;
Afdel, Karim .
MULTIMEDIA TOOLS AND APPLICATIONS, 2020, 79 (29-30) :20483-20518
[5]   Multi-scale CNN based on region proposals for efficient breast abnormality recognition [J].
Bakkouri, Ibtissam ;
Afdel, Karim .
MULTIMEDIA TOOLS AND APPLICATIONS, 2019, 78 (10) :12939-12960
[6]  
Bell-Kligler S, 2019, ADV NEUR IN, V32
[7]   Single Image Super-Resolution via a Holistic Attention Network [J].
Niu, Ben ;
Wen, Weilei ;
Ren, Wenqi ;
Zhang, Xiangde ;
Yang, Lianping ;
Wang, Shuzhen ;
Zhang, Kaihao ;
Cao, Xiaochun ;
Shen, Haifeng .
COMPUTER VISION - ECCV 2020, PT XII, 2020, 12357 :191-207
[8]   To Learn Image Super-Resolution, Use a GAN to Learn How to Do Image Degradation First [J].
Bulat, Adrian ;
Yang, Jing ;
Tzimiropoulos, Georgios .
COMPUTER VISION - ECCV 2018, PT VI, 2018, 11210 :187-202
[9]   Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model [J].
Cai, Jianrui ;
Zeng, Hui ;
Yong, Hongwei ;
Cao, Zisheng ;
Zhang, Lei .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :3086-3095
[10]   Camera Lens Super-Resolution [J].
Chen, Chang ;
Xiong, Zhiwei ;
Tian, Xinmei ;
Zha, Zheng-Jun ;
Wu, Feng .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :1652-1660