Remote Sensing Image Fusion Based on Two-Stream Fusion Network

Remote sensing image fusion (or pan-sharpening) aims at generating high resolution multi-spectral (MS) image from inputs of a high spatial resolution single band panchromatic (PAN) image and a low spatial resolution multi-spectral image. In this paper, a deep convolutional neural network with two-stream inputs respectively for PAN and MS images is proposed for remote sensing image pan-sharpening. Firstly the network extracts features from PAN and MS images, then it fuses them to form compact feature maps that can represent both spatial and spectral information of PAN and MS images, simultaneously. Finally, the desired high spatial resolution MS image is recovered from the fused features using an encoding-decoding scheme. Experiments on Quickbird satellite images demonstrate that the proposed method can fuse the PAN and MS image effectively.