Mamba Collaborative Implicit Neural Representation for Hyperspectral and Multispectral Remote Sensing Image Fusion

被引:8
作者
Zhu, Chunyu [1 ]
Deng, Shangqi [2 ]
Song, Xuan [3 ]
Li, Yachao [4 ]
Wang, Qi [1 ]
机构
[1] Xidian Univ, Hangzhou Inst Technol, Hangzhou 311231, Zhejiang, Peoples R China
[2] Xi An Jiao Tong Univ, Natl Key Lab Human Machine Hybrid Augmented Intell, Natl Engn Res Ctr Visual Informat & Applicat, Inst Artificial Intelligence & Robot, Xian 710049, Shaanxi, Peoples R China
[3] Northwestern Polytech Univ, Sch Astronaut, Xian 710072, Peoples R China
[4] Xidian Univ, Natl Key Lab Radar Signal Proc, Xian 710071, Shaanxi, Peoples R China
来源
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING | 2025年 / 63卷
基金
中国国家自然科学基金;
关键词
Feature extraction; Accuracy; Spatial resolution; Matrix decomposition; Optimization; Deep learning; Sparse approximation; Image fusion; Hyperspectral imaging; Transformers; Hyperspectral and multispectral image fusion; implicit neural representation (INR); Mamba; Mamba cooperative INR fusion network (MCIFNet); state-space model (SSM); SUPERRESOLUTION; NETWORK; SPARSE; MULTISCALE;
D O I
10.1109/TGRS.2025.3537638
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Hyperspectral remote sensing images (HSIs) capture detailed spectral characteristics of features, while multi- spectral remote sensing images (MSIs) provide clear spatial distribution. Fusing these two types of images can enhance feature identification and classification accuracy. Current deep learning algorithms achieve high fusion quality but struggle with balancing global effective perception and lightweight computation. Moreover, these algorithms typically discretely handle data mapping, which contrasts with the continuous nature of the world. Recently, the Mamba has shown significant potential for complex long-range modeling, addressing the computational complexity of global perception. Concurrently, implicit neural representation (INR) offers high-quality solutions for continuous domain modeling. To this end, this study introduces a novel network architecture that combines Mamba and INR, termed the Mamba cooperative INR fusion network (MCIFNet). MCIFNet effectively captures global image information and generates fused images in a continuous domain through pointto-point processing. The network comprises two main units: potential space projection and semantic extraction and fusion. The potential space projection unit performs shallow encoding of hyperspectral and MSIs, mapping them to a latent feature space. The semantic extraction and fusion unit (SEFU) uses scale adaptive residual state spatial and implicit spatial-spectral fusion (ISSF) modules to extract deep features from the bimodal images, generating fused images point-by-point. A series of fusion experiments with 4x, 8x, and 16x scale factors demonstrate that MCIFNet surpasses popular algorithms in both spatial detail and spectral information reconstruction, while also providing more lightweight performance. The code for MCIFNet will be shared on https://github.com/chunyuzhu/MCIFNet.
引用
收藏
页数:15
相关论文
共 57 条
[51]   Hyperspectral and Multispectral Image Fusion Based on Local Low Rank and Coupled Spectral Unmixing [J].
Zhou, Yuan ;
Feng, Liyang ;
Hou, Chunping ;
Kung, Sun-Yuan .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2017, 55 (10) :5997-6009
[52]   An Implicit Transformer-based Fusion Method for Hyperspectral and Multispectral Remote Sensing Image [J].
Zhu, Chunyu ;
Zhang, Tinghao ;
Wu, Qiong ;
Li, Yachao ;
Zhong, Qin .
INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION, 2024, 131
[53]   An adaptive multi-perceptual implicit sampling for hyperspectral and multispectral remote sensing image fusion [J].
Zhu, Chunyu ;
Dai, Rongyuan ;
Gong, Liwei ;
Gao, Liangbo ;
Ta, Na ;
Wu, Qiong .
INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION, 2023, 125
[54]   QIS-GAN: A Lightweight Adversarial Network With Quadtree Implicit Sampling for Multispectral and Hyperspectral Image Fusion [J].
Zhu, Chunyu ;
Deng, Shangqi ;
Zhou, Yingjie ;
Deng, Liang-Jian ;
Wu, Qiong .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2023, 61 :1-15
[55]   MGDIN: Detail Injection Network for HSI and MSI Fusion Based on Multiscale and Global Contextual Features [J].
Zhu, Chunyu ;
Gong, Liwei ;
Zhang, Ying ;
Chen, Shengbo ;
Gao, Liangbo ;
Ta, Na ;
Wu, Qiong .
INTERNATIONAL JOURNAL OF REMOTE SENSING, 2023, 44 (18) :5574-5596
[56]   Hyperspectral and multispectral remote sensing image fusion using SwinGAN with joint adaptive spatial-spectral gradient loss function [J].
Zhu, Chunyu ;
Deng, Shangqi ;
Li, Jiaxin ;
Zhang, Ying ;
Gong, Liwei ;
Gao, Liangbo ;
Ta, Na ;
Chen, Shengbo ;
Wu, Qiong .
INTERNATIONAL JOURNAL OF DIGITAL EARTH, 2023, 16 (01) :3580-3600
[57]  
Zhu QF, 2024, Arxiv, DOI [arXiv:2404.01705, DOI 10.1016/J.HELIYON.2024.E38495]