Deep Neural Network Based Hyperspectral Pixel Classification With Factorized Spectral-Spatial Feature Representation

Deep learning has been widely used for hyperspectral pixel classification due to its ability to generate deep feature representation. However, how to construct an efficient and powerful network suitable for hyperspectral data is still under exploration. In this paper, a novel neural network model is designed for taking full advantage of the spectral-spatial structure of hyperspectral data. First, we extract pixel-based intrinsic features from rich yet redundant spectral bands by a subnetwork with the supervised pretraining scheme. Second, in order to utilize the local spatial correlation among pixels, we share the previous subnetwork as a spectral feature extractor for each pixel in a patch of the image, after which the spectral features of all pixels in a patch are combined and fed into the subsequent classification subnetwork. Finally, the whole network is further fine-tuned to improve its classification performance. Especially, the spectralspatial factorization scheme is applied in our model architecture, making the network size and the number of parameters great less than the existing spectral-spatial deep networks for hyperspectral image classification. Compared with other state-of-the-art deep learning methods, experiments on the hyperspectral data sets show that our method achieves 0.18%-7.6%, 0.1%-3.58%, and 0.21%-3.09% improvement on overall accuracy (OA), average accuracy (AA), and kappa, respectively, while having smaller network size and fewer parameters.