Pyramid-attention based multi-scale feature fusion network for multispectral pan-sharpening

Remote sensing images with high spatial resolution and high spectral resolution have important applications in human society. In general, due to the limitations faced by the optical sensors’, we are limited to obtain only low spatial resolution multispectral images (MS) and high spatial resolution panchromatic images (PAN). To address this limitation, this study proposes a pyramid-attention based multi-scale feature fusion network (PAMF-Net) that combines the pyramid attention mechanism and feature aggregation. Initially, the MS and PAN images are input to the network, and the PAN images pass through the input pyramid branch to generate a multi-level receiving domain. Then, the result is combined with the features of the MS image as the input of the encoder, and these composite features are input to the pyramid attention mechanism module to capture multi-scale corresponding features. Next, the result of the input pyramid branch is input to the feature aggregation module to seamlessly merge with the features of the pyramid attention mechanism. Finally, in the encoding stage, multiple levels of features are multiplexed as encoding secondary lines by skipping connections to obtain high-quality HRMS images. After quantitative and qualitative experiments, the results show that our method is superior to other advanced methods.