PSSTFN: A Progressive Spatial-Temporal-Spectral Fusion Network for Remote Sensing Images

Spatial-temporal-spectral fusion (STSF) is highly desirable to generate dense-time image series with high spatial and spectral resolution by integrating the complementary advantages of multisource and multitemporal observations. However, most existing STSF methods are still limited to the assumption of linear temporal, spatial, and spectral relationships. In addition, the STSF methods on Landsat and MODIS data are insufficient to characterize the inherent properties of the current spaceborne hyperspectral (HS) images with low spatial and temporal resolutions. For these, we propose a progressive STSF network (PSSTFN) by interestingly integrating spatial-spectral fusion and spectral-temporal fusion into a unified end-to-end STSF framework. Specifically, in the spatial-spectral fusion stage, we obtain the hierarchical features with different receptive fields and propose a multiattention guided (MAG) module for joint learning and refinement of spatial-spectral features. In the spectral-temporal fusion stage, a feature insertion module (FIM) is presented to embed the difference images into the resulting spatial-spectral features, and the estimation from deeper layers is cascaded for more reliable spatial information. We build the Dongying (DY) and Yellow River Estuary (YRE) remote sensing datasets based on Sentinel-2 and ZiYuan (ZY)-1 02D satellites for verification, and the experimental results on reduce- and full-resolution data demonstrate the superior performance of our method over the existing methods visually and quantitatively.