Small Sample Hyperspectral Image Classification Based on Cascade Fusion of Mixed Spatial-Spectral Features and Second-Order Pooling

Hyperspectral images can capture subtle differences in reflectance of features in hundreds of narrow bands, and its pixel-wise classification is the cornerstone of many applications requiring fine-grained classification results. Although three-dimensional convolutional neural networks (3D-CNN) have been extensively investigated in hyperspectral image classification tasks and have made significant breakthroughs, hyperspectral classification under small sample conditions is still challenging. In order to facilitate small sample hyperspectral classification, a novel mixed spatial-spectral features cascade fusion network (MSSFN) is proposed. First, the covariance structure of hyperspectral data is modeled and dimensionality reduction is conducted using factor analysis. Then, two 3D spatial-spectral residual modules and one 2D separable spatial residual module are used to extract mixed spatial-spectral features. A cascade fusion pattern consisting of intra-block feature fusion and inter-block feature fusion is constructed to enhance the feature extraction capability. Finally, the second-order statistical information of the fused features is mined using second-order pooling and the classification is achieved by the fully connected layer after L2 normalization. On the three public available hyperspectral datasets, Indian Pines, Houston, and University of Pavia, only 5%, 3%, and 1% of the labeled samples were used for training, the accuracy of MSSFN in this paper is 98.52%, 96.31% and 98.83%, respectively, which is far better than the contrast models and verifies the effectiveness of MSSFN in small sample hyperspectral classification tasks.