SMFE-Net: a saliency multi-feature extraction framework for VHR remote sensing image classification

Scene classification of very-high resolution (VHR) remote sensing images is a challenging research hotspot. It is difficult to extract salient features because of the characteristics of remote sensing images, such as large spatial range changes and complex scenes. In addition, the effective combination of high-level semantic information and low-level contour information is also a major difficulty at present. In order to solve these problems, we proposed a new end-to-end saliency multi-feature extraction network (SMFE-Net) based on VGG16 and long short-term memory (LSTM) to extract salient features and effectively integrate high-level features with low-level features. Firstly, we design an adaptive memory network (AMN) based on the rectangular combination of LSTMs to capture rich features of high and low levels. The AMN not only provides supplementary information but also focuses on key areas, thus discarding non-critical information. Secondly, in order to realize adaptive feature extraction, the sequential connection of channel attention (CA) and spatial attention (SA) is placed in the high-level feature extraction subnetwork, whose outputs are multiplied by the weights of the last feature map layer of VGG16. Finally, the outputs of AMN and the attention-weighted features are concatenated and inputted to the fully connected layer for the scene classification of the VHR remote sensing image. To verify the validity of the proposed SMFE-Net, the UC Merced (UCM) land-use dataset, the Aerial Image Dataset (AID), and the OPTIMAL-31 (OPTL) dataset are selected as the experimental materials. Experimental results have demonstrated that the proposed SMFE-Net is superior to several most advanced methods.