Dual-Task Cascaded Network for Spatial-Temporal-Spectral Remote Sensing Image Fusion

Spatial-temporal-spectral fusion (STSF) is dedicated to integrating the complementary advantages of multisource images to obtain fused images with all high spatial, high temporal, and high spectral resolutions, which is promising but more challenging. On the one hand, traditional studies deployed on MODIS and Landsat data cannot be transferred to most spaceborne hyperspectral (HS) data with lower temporal resolution; on the other hand, the rigid time relation modeling in most existing studies exhibits weakness orienting to nonlinear land-cover changes. In this article, we propose a dual-task cascaded network for STSF (DC-STSF), with collaborative modeling on spatial-spectral joint enhancement (SSJE) and temporal variation estimation (TVE) in a unified framework. The SSJE task was designed with an iterative alternating projection, meticulously crafted to address the scale variance among observations. Additionally, the spatial enhancement unit (SEU) and error correction unit (ECU) were coupled modeling to enhance the spatial and spectral fidelities. The TVE on the spectral fine-tuning network (SFTN) was developed, to further enhance the temporal and spectral fidelities. Extensive experiments were implemented on Ziyuan (ZY)-1 02D HS data and Sentinel-2 multispectral (MS) data. Both qualitative and quantitative results demonstrated the competitive performance of the proposed method.