Evaluating the potential of pyramid-based fusion coupled with convolutional neural network for satellite image classification

Deep learning (DL)-based methods have recently been extensively used for satellite image analysis due to their ability to automatically extract spatial-spectral features from images. Recent advancement in DL-based methods has also allowed the remote sensing community to utilize these methods in fusing the satellite images for enhanced land use/land cover (LULC) classification. Keeping this in view, the present study aims to evaluate the potential of SAR (Sentinel 1) and Optical (Sentinel 2) image fusion using pyramid-based DL methods over an agricultural area in India. In this study, three image fusion methods, i.e., pyramid-based fusion methods, pyramid-based fusion methods coupled with convolutional neural network (CNN), and a combination of two different pyramid decomposition methods concurrently with CNN were used. The performance of the fused images was evaluated in terms of fusion metrics, image quality, and overall classification accuracy by an optimized 2D-CNN-based DL classifier. Results from pyramid-based fusion methods with CNN and a combination of two different pyramid decomposition methods with CNN suggest that these methods were able to retain visual quality as well as the detailed structural information of input images in comparison to the pyramid-based fusion methods. Bayesian optimization method was used to optimize various hyper-parameters of the 2D-CNN-based DL classifier used in this study. Results with fused images obtained using pyramid-based methods coupled with CNN suggest an improved performance by VV (Vertical–Vertical) polarized images in terms of overall classification accuracy (99.23% and 99.33%).