Mamba Collaborative Implicit Neural Representation for Hyperspectral and Multispectral Remote Sensing Image Fusion

被引:0
作者
Zhu, Chunyu [1 ]
Deng, Shangqi [2 ]
Song, Xuan [3 ]
Li, Yachao [4 ]
Wang, Qi [1 ]
机构
[1] Xidian Univ, Hangzhou Inst Technol, Hangzhou 311231, Zhejiang, Peoples R China
[2] Xi An Jiao Tong Univ, Natl Key Lab Human Machine Hybrid Augmented Intell, Natl Engn Res Ctr Visual Informat & Applicat, Inst Artificial Intelligence & Robot, Xian 710049, Shaanxi, Peoples R China
[3] Northwestern Polytech Univ, Sch Astronaut, Xian 710072, Peoples R China
[4] Xidian Univ, Natl Key Lab Radar Signal Proc, Xian 710071, Shaanxi, Peoples R China
来源
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING | 2025年 / 63卷
基金
中国国家自然科学基金;
关键词
Feature extraction; Accuracy; Spatial resolution; Matrix decomposition; Optimization; Deep learning; Sparse approximation; Image fusion; Hyperspectral imaging; Transformers; Hyperspectral and multispectral image fusion; implicit neural representation (INR); Mamba; Mamba cooperative INR fusion network (MCIFNet); state-space model (SSM); SUPERRESOLUTION; NETWORK; SPARSE; MULTISCALE;
D O I
10.1109/TGRS.2025.3537638
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Hyperspectral remote sensing images (HSIs) capture detailed spectral characteristics of features, while multi- spectral remote sensing images (MSIs) provide clear spatial distribution. Fusing these two types of images can enhance feature identification and classification accuracy. Current deep learning algorithms achieve high fusion quality but struggle with balancing global effective perception and lightweight computation. Moreover, these algorithms typically discretely handle data mapping, which contrasts with the continuous nature of the world. Recently, the Mamba has shown significant potential for complex long-range modeling, addressing the computational complexity of global perception. Concurrently, implicit neural representation (INR) offers high-quality solutions for continuous domain modeling. To this end, this study introduces a novel network architecture that combines Mamba and INR, termed the Mamba cooperative INR fusion network (MCIFNet). MCIFNet effectively captures global image information and generates fused images in a continuous domain through pointto-point processing. The network comprises two main units: potential space projection and semantic extraction and fusion. The potential space projection unit performs shallow encoding of hyperspectral and MSIs, mapping them to a latent feature space. The semantic extraction and fusion unit (SEFU) uses scale adaptive residual state spatial and implicit spatial-spectral fusion (ISSF) modules to extract deep features from the bimodal images, generating fused images point-by-point. A series of fusion experiments with 4x, 8x, and 16x scale factors demonstrate that MCIFNet surpasses popular algorithms in both spatial detail and spectral information reconstruction, while also providing more lightweight performance. The code for MCIFNet will be shared on https://github.com/chunyuzhu/MCIFNet.
引用
收藏
页数:15
相关论文
共 57 条
[51]  
Chen K., Chen B., Liu C., Li W., Zou Z., Shi Z., RSMamba: Remote sensing image classification with state space model, IEEE Geosci. Remote Sens. Lett., 21, pp. 1-5, (2024)
[52]  
Hu J., Shen L., Sun G., Squeeze-and-excitation networks, Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., pp. 7132-7141, (2018)
[53]  
Zhao H., Gallo O., Frosio I., Kautz J., Loss functions for image restoration with neural networks, IEEE Trans. Comput. Imag., 3, 1, pp. 47-57, (2017)
[54]  
Wald L., Ranchin T., Mangolini M., Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images, Photogramm. Eng. Remote Sens., 63, 6, pp. 691-699, (1997)
[55]  
Yokoya N., Yairi T., Iwasaki A., Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion, IEEE Trans. Geosci. Remote Sens., 50, 2, pp. 528-537, (2011)
[56]  
Simoes M., Bioucas-Dias J., Almeida L.B., Chanussot J., A convex formulation for hyperspectral image superresolution via subspace-based regularization, IEEE Trans. Geosci. Remote Sens., 53, 6, pp. 3373-3388, (2015)
[57]  
Cao Z., Wu X., Deng L.-J., Zhong Y., A novel state space model with local enhancement and state sharing for image fusion, (2024)