SRSF-GAN: A Super-Resolution-Based Spatial Fusion With GAN for Satellite Images With Different Spatial and Temporal Resolutions

Recently, spatiotemporal fusion technologies have been rapidly developed and widely applied, which generally require one or more pairs of coarse- and fine-resolution images as reference data and a coarse-resolution image at the prediction time to produce a fine-resolution image at the forecast time. Consequently, most spatiotemporal fusion methods are phase-based and obtain temporal changing information from the coarse-resolution image pairs. Consequently, they usually have rigid constraints on reference data selection using the temporal interval criterion. However, due to the relatively long revisit cycle and cloud contamination, it is difficult to prepare adequate high-quality reference data with little change, especially for large-scale fusion tasks. Therefore, we propose a spatial-based fusion method that only requires the coarse images at the prediction time and a fine reference image selected by a spatial-spectral-similarity criterion, named super-resolution-based spatial fusion with the generative adversarial network (SRSF-GAN). SRSF-GAN uses a super-resolution (SR) module merely on the coarse images at the prediction time and then performs a multiscale fusion with the reference fine image. Moreover, the spatial attention mechanism is adopted to achieve dynamic weight tuning, i.e., assigning more weight to the SR image for changed areas and more weight to the fine reference image for unchanged areas. Comparison experiments based on three datasets show that our model can outperform the state-of-the-art methods, and the changes with different spatial and spectral ranges of variation can be recovered. The code will be uploaded to the following website: https://github.com/Zhaosir996/SRSFGAN.