Development status and future prospects of multi-source remote sensing image fusion

The development of multispectral, hyperspectral, infrared, radar, and other sensing technologies in recent years has facilitated the use of remote sensing methods in precision agriculture, resource investigation, environmental monitoring, military defense, and other fields. Multi-source remote sensing images in the same scene can capture the same ground objects, while the dimensions of the observations are independent of each other. Therefore, the imaging scale, spatial resolution, time resolution, and target characteristics may be quite different in different observations. The information provided by massive multi-source remote sensing data is redundant, complementary, and cooperative. Multi-source remote sensing image fusion can utilize the complementary information obtained from different sources to achieve accurate and comprehensive Earth observations. Thus, it is one of the key technologies in remote sensing. From the perspective of data sources, this review summarizes the research status and future development trends of multi-source remote sensing image fusion. In the introduction, the importance of multi-source image fusion and the motivation of this review are illustrated briefly. The second section outlines the main sources and image characteristics of nine typical remote sensing data: panchromatic images, multispectral images, hyperspectral images, infrared images, nighttime light images, stereo images, video images, Synthetic Aperture Radar (SAR) images, and light detection and ranging (LiDAR) images. The typical applications of these multi-source data are also briefly concluded while introducing the characteristics of these multi-source remote sensing images separately. Moreover, the development trend of multi-source remote sensing image fusion is evaluated according to the number of publications. In the third section, latest studies on multi-source remote sensing image fusion are introduced in detail in the order of optical image fusion, optical and SAR image fusion, optical and LiDAR image fusion, and other types of remote sensing image fusion. The third section also puts forward some challenging problems in remote sensing image fusion. For example, the registration problem of multi-source images, the application problem of fusion in specific domain, and the representation of features during cross-modal fusion are all important problems that need to be solved urgently. In the conclusion section, this review summarizes the research status of the multi-source remote sensing image fusion field. This also section prospects the future development trend of multi-source remote sensing image fusion. First, the study of related fusion technologies for new types of remote sensing images will be a major future research. Second, the integration of data acquisition and image fusion techniques can reduce the difficulty and improve the performance of image fusion with the help of novel hardware designs. Therefore, multi-modal fusion-based computational imaging systems should be designed. Third, fusing multi-source images with other types of data, such as geographical, ground station, and web data, is an interesting research topic in addition to the fusion of remote sensing images. Finally, evaluating the performance of image fusion is an important problem. Image fusion aims to help better understand the land covers from different dimensions of Earth observations. Whether the fusion can help the understanding of the Earth is unclear. Therefore, the improvement in application performance, such as detection or classification accuracy, may be an important index compared with the enhancement in the quality of the fused image. © 2021, Science Press. All right reserved.