MSTSENet: Multiscale Spectral-Spatial Transformer with Squeeze and Excitation network for hyperspectral image classification

Hyperspectral image (HSI) classification pertains to the task of assigning a single label to each pixel by analyzing its spectral-spatial characteristics. Convolutional Neural Networks (CNNs) have garnered significant attention on account of their remarkable performance in feature representation. However, these approaches possess a restricted capacity to acquire highly similar spectral features. Recently, Attention -based approaches have been devised as a means to surmount the limitations inherent in CNNs. In this paper, we address a novel architecture for spectral-spatial feature extraction and HSI classification. The framework efficiently amalgamates the robustness and efficacy of three modern backbone networks, i.e., Multiscale CNN, Squeeze and Excitation (SE), and Transformer named MSTSENet. Initially, the multiscale convolution module integrates three branches of 3D convolution layers, each employing distinct kernel sizes, facilitating multiscale and multi -resolution feature extraction. Subsequently, the SE module enhances the inter -channel relationship by adaptively recalibrating the feature weight. The strategic fusion of the SE module with multiscale CNN strengthens deep feature extraction while introducing a minimal increment in overall parameters. Lastly, the Transformer module orchestrates information aggregation across different spectral-spatial regions, accurately modeling long-range contextual dependencies and capturing dominant feature representations. To evaluate the model performance, comprehensive experiments have been carried out on widely used benchmark HSI datasets. The overall accuracy of 98.74%, 99.83%, and 99.98% is achieved over the Indian Pines, Pavia University, and Salinas Valley datasets, respectively. The code of this work will be accessible at https: //github.com/irfan01000/MSTSENet.