Hyperspectral and multispectral image fusion: When model-driven meet data-driven strategies

被引:1
作者
Yan, Hao-Fang [1 ]
Zhao, Yong-Qiang [1 ]
Chan, Jonathan Cheung-Wai [2 ]
Kong, Seong G. [3 ]
EI-Bendary, Nashwa [4 ]
Reda, Mohamed [5 ]
机构
[1] Northwestern Polytech Univ, Sch Automat, Xian 710072, Peoples R China
[2] Vrije Univ Brussel, Dept Elect & Informat, B-1050 Brussels, Belgium
[3] Sejong Univ, Dept Comp Engn, Seoul 05006, South Korea
[4] Arab Acad Sci Technol & Maritime Transport AASTMT, Coll Comp & Informat Technol, Kerdasa, Egypt
[5] Mil Tech Coll, Cairo 11765, Egypt
基金
中国国家自然科学基金;
关键词
Technical review; Hyperspectral and multispectral image fusion; Model-driven; Data-driven; Model-data-driven; TENSOR FACTORIZATION; MAP ESTIMATION; SUPERRESOLUTION; MULTIRESOLUTION; DECOMPOSITION; NETWORK; CLASSIFICATION; REPRESENTATION; TRANSFORM; INVERSION;
D O I
10.1016/j.inffus.2024.102803
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Hyperspectral image (HSI) and Multispectral Image (MSI) fusion aims at combining a high-resolution MSI (HR MSI) with a low-resolution HSI (LR HSI), resulting in a fused image that contains the spatial resolution of the former and the spectral resolution of the latter. This approach offers a cost-effective alternative to directly acquiring high-resolution HSIs (HR HSIs). In this survey, we offer an extensive literature review tailored for students and professionals seeking deeper insights into the subject matter. We delve into existing HSI-MSI fusion methods and revealed a spectrum of approaches, ranging from model-driven techniques (extended CS and MRA, Bayesian, matrix factorization, and tensor representation) to data-driven methods (CNN, GAN, and Transformer) and model-data-driven approaches (model-guided networks and semi-supervised or unsupervised methods). This exploration aims to optimize fusion strategies for various applications. This paper not only provides a comprehensive overview of HSI-MSI fusion strategies, but also summarizes and contrasts their unique characteristics, benefits, and limitations. Additionally, it reviews image quality evaluation indices (both full-reference and noreference) and widely used datasets. Furthermore, using hybrid data, large-view-field satellite data and real satellite data pairs, the reduced-resolution and full-resolution experimental comparison analysis of various algorithms from three strategies are carried out. Finally, the paper identifies unresolved challenges and outlines potential future research directions in this evolving field.
引用
收藏
页数:24
相关论文
共 201 条
[21]   Unsupervised Hybrid Network of Transformer and CNN for Blind Hyperspectral and Multispectral Image Fusion [J].
Cao, Xuheng ;
Lian, Yusheng ;
Wang, Kaixuan ;
Ma, Chao ;
Xu, Xianqing .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 :1-15
[22]  
CARPER WJ, 1990, PHOTOGRAMM ENG REM S, V56, P459
[23]   Weighted Low-Rank Tensor Recovery for Hyperspectral Image Restoration [J].
Chang, Yi ;
Yan, Luxin ;
Zhao, Xi-Le ;
Fang, Houzhang ;
Zhang, Zhijun ;
Zhong, Sheng .
IEEE TRANSACTIONS ON CYBERNETICS, 2020, 50 (11) :4558-4572
[24]  
CHAVEZ PS, 1989, PHOTOGRAMM ENG REM S, V55, P339
[25]  
CHAVEZ PS, 1991, PHOTOGRAMM ENG REM S, V57, P295
[26]   Trifocal Tensor-Based Adaptive Visual Trajectory Tracking Control of Mobile Robots [J].
Chen, Jian ;
Jia, Bingxi ;
Zhang, Kaixiang .
IEEE TRANSACTIONS ON CYBERNETICS, 2017, 47 (11) :3784-3798
[27]   SFA-guided mosaic transformer for tracking small objects in snapshot spectral imaging [J].
Chen, Lulu ;
Zhao, Yongqiang ;
Kong, Seong G. .
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, 2023, 204 :223-236
[28]   Histograms of oriented mosaic gradients for snapshot spectral image description [J].
Chen, Lulu ;
Zhao, Yongqiang ;
Chan, Jonathan Cheung-Wai ;
Kong, Seong G. .
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, 2022, 183 :79-93
[29]   FTDN: Multispectral and Hyperspectral Image Fusion With Diverse Temporal Difference Spans [J].
Chen, Xu ;
Meng, Xiangchao ;
Liu, Qiang ;
Jiang, Huiping ;
Yang, Gang ;
Sun, Weiwei ;
Shao, Feng .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2023, 61
[30]   Fusion of Hyperspectral and Multispectral Images: A Novel Framework Based on Generalization of Pan-Sharpening Methods [J].
Chen, Zhao ;
Pu, Hanye ;
Wang, Bin ;
Jiang, Geng-Ming .
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2014, 11 (08) :1418-1422