Fusion of ConvLSTM and Multi-Attention Mechanism Network for Hyperspectral Image Classification

In recent years, deep learning-based models have achieved remarkable results in the hyperspectral image ( HSI) classification. Aiming at the low classification accuracy of deep learning-based HSI classification methods under limited sample data, this paper proposes an HSI classification method that combines ConvLSTM and a multi-attention mechanism network. The method is divided into three branches: spectral branch, spatial-X branch and spatial-Y branch to extract spectral features, spatial-X features and spatial-Y features respectively, and fuse the features in three directions for hyperspectral image classification. Since convolutional long short-term memory (ConvLSTM) shows good performance in learning valuable features and modeling long-term dependencies in spectral data, 3 hidden layers are used in the spectral branch, and the convolution kernel size is 3X 3, the channels are 150, 100 and 60, respectively, to extract spectral information. On the spatial-X and spatial-Y branches, Dense spatial-X blocks and Dense spatial-Y blocks based on DenseNet and 3D-CNN are used to extract spatial-X and spatial-Y features, respectively. In order to enhance feature extraction, the attention mechanism of its feature direction is also introduced in these three branches, respectively. The spectral attention blocks are designed for the information-rich spectral bands, and a spatial-X attention block and a spatial-Y attention block are designed for the information-rich pixels, respectively. Experiments were conducted on three publicly available hyperspectral datasets, namely Indian Pines (IP) Pavia University (UP) and Salinas Valley (SV) datasets, and compared with five other methods: the SVM with RBF kernel (SVM), Going Deeper with Contextual CNN (CDCNN), Fast Dense Spectral-Spatial Convolution (FDSSC), Spectral-Spatial Residual Network (SSRN), Double-Branch Dual-Attention Mechanism Network (DBDA). In the experiments, the size of training and validation samples is set to 3% of the total samples on the IP dataset, and 0. 5% of the total samples on the UP and SV datasets. For our method and all deep learning-based methods, the batch size is set to 16, the optimizer is set to Adam, the learning rate is set to 0. 000 5, and the learning rate is dynamically adjusted. Since SVM directly uses spectral information for classification, the pixel size of the input sample block is 1 X 1, and the pixels of other input sample blocks based on deep learning methods are all set to 9 X 9. The experimental results show that the method in this paper can fully use the spectral and spatial characteristics of HSI, and achieve better results in the evaluation criteria such as OA, AA, and KAPPA. Among them, the OA index of the method in this paper is improved by 0. 12%0-2. 04% on average compared with the suboptimal algorithm.