A Multi-Stage Progressive Pansharpening Network Based on Detail Injection with Redundancy Reduction

被引:2
作者
Wen, Xincan [1 ,2 ]
Ma, Hongbing [1 ,2 ,3 ]
Li, Liangliang [4 ]
机构
[1] Xinjiang Univ, Sch Comp Sci & Technol, Urumqi 830046, Peoples R China
[2] Xinjiang Univ, Key Lab Signal Detect & Proc, Urumqi 830046, Peoples R China
[3] Tsinghua Univ, Dept Elect Engn, Beijing 100084, Peoples R China
[4] Beijing Inst Technol, Sch Informat & Elect, Beijing 100081, Peoples R China
关键词
deep learning; pansharpening; feature extraction; image reconstruction; multi-stage; FUSION; IMAGES;
D O I
10.3390/s24186039
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
In the field of remote sensing image processing, pansharpening technology stands as a critical advancement. This technology aims to enhance multispectral images that possess low resolution by integrating them with high-spatial-resolution panchromatic images, ultimately producing multispectral images with high resolution that are abundant in both spatial and spectral details. Thus, there remains potential for improving the quality of both the spectral and spatial domains of the fused images based on deep-learning-based pansharpening methods. This work proposes a new method for the task of pansharpening: the Multi-Stage Progressive Pansharpening Network with Detail Injection with Redundancy Reduction Mechanism (MSPPN-DIRRM). This network is divided into three levels, each of which is optimized for the extraction of spectral and spatial data at different scales. Particular spectral feature and spatial detail extraction modules are used at each stage. Moreover, a new image reconstruction module named the DRRM is introduced in this work; it eliminates both spatial and channel redundancy and improves the fusion quality. The effectiveness of the proposed model is further supported by experimental results using both simulated data and real data from the QuickBird, GaoFen1, and WorldView2 satellites; these results show that the proposed model outperforms deep-learning-based methods in both visual and quantitative assessments. Among various evaluation metrics, performance improves by 0.92-18.7% compared to the latest methods.
引用
收藏
页数:24
相关论文
共 63 条
[1]   MTF-tailored multiscale fusion of high-resolution MS and pan imagery [J].
Aiazzi, B. ;
Alparone, L. ;
Baronti, S. ;
Garzelli, A. ;
Selva, M. .
PHOTOGRAMMETRIC ENGINEERING AND REMOTE SENSING, 2006, 72 (05) :591-596
[2]   Multispectral and panchromatic data fusion assessment without reference [J].
Alparone, Luciano ;
Alazzi, Bruno ;
Baronti, Stefano ;
Garzelli, Andrea ;
Nencini, Filippo ;
Selva, Massimo .
PHOTOGRAMMETRIC ENGINEERING AND REMOTE SENSING, 2008, 74 (02) :193-200
[3]   A variational model for P+XS image fusion [J].
Ballester, Coloma ;
Caselles, Vicent ;
Igual, Laura ;
Verdera, Joan ;
Rougé, Bernard .
INTERNATIONAL JOURNAL OF COMPUTER VISION, 2006, 69 (01) :43-58
[4]   Super-Resolution-Guided Progressive Pansharpening Based on a Deep Convolutional Neural Network [J].
Cai, Jiajun ;
Huang, Bo .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2021, 59 (06) :5206-5220
[5]   GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond [J].
Cao, Yue ;
Xu, Jiarui ;
Lin, Stephen ;
Wei, Fangyun ;
Hu, Han .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW), 2019, :1971-1980
[6]   Multisensor Satellite Image Fusion and Networking for All-Weather Environmental Monitoring [J].
Chang, Ni-Bin ;
Bai, Kaixu ;
Imen, Sanaz ;
Chen, Chi-Farn ;
Gao, Wei .
IEEE SYSTEMS JOURNAL, 2018, 12 (02) :1341-1357
[7]   Image Fusion with Local Spectral Consistency and Dynamic Gradient Sparsity [J].
Chen, Chen ;
Li, Yeqing ;
Liu, Wei ;
Huang, Junzhou .
2014 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2014, :2760-2765
[8]   Slice-to-slice context transfer and uncertain region calibration network for shadow detection in remote sensing imagery [J].
Chen, Hongyu ;
Feng, Dejun ;
Cao, Shaohan ;
Xu, Wanqi ;
Xie, Yakun ;
Zhu, Jun ;
Zhang, Heng .
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, 2023, 203 :166-182
[9]   Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks [J].
Chen, Jierun ;
Kao, Shiu-Hong ;
He, Hao ;
Zhuo, Weipeng ;
Wen, Song ;
Lee, Chul-Ho ;
Chan, S. -H. Gary .
2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2023, :12021-12031
[10]   TVConv: Efficient Translation Variant Convolution for Layout-aware Visual Processing [J].
Chen, Jierun ;
He, Tianlang ;
Zhuo, Weipeng ;
Ma, Li ;
Ha, Sangtae ;
Chan, S-H Gary .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, :12538-12548