Hybrid Multiscale Spatial-Spectral Transformer for Hyperspectral Image Classification

Hyperspectral image (HSI) classification constitutes a significant foundation for remote sensing analysis. Transformer architecture establishes long-range dependencies with a self-attention mechanism (SA), which exhibits advantages in HSI classification. However, most existing transformer-based methods are inadequate in exploring the multiscale properties of hybrid spatial and spectral information inherent in HSI data. To countermeasure this problem, this work investigates a hybrid multiscale spatial-spectral framework (HMSSF). It innovatively models global dependencies across multiple scales from both spatial and spectral domains, which allows for cooperatively capturing hybrid multiscale spatial and spectral characteristics for HSI classification. Technically, a spatial-spectral token generation (SSTG) module is first designed to generate the spatial tokens and spectral tokens. Then, a multiscale SA (MSSA) is developed to achieve multiscale attention modeling by constructing different dimensional attention heads per attention layer. This mechanism is adaptively integrated into both spatial and spectral branches for hybrid multiscale feature extraction. Furthermore, a spatial-spectral attention aggregation (SSAA) module is introduced to dynamically fuse the multiscale spatial and spectral features to enhance the classification robustness. Experimental results and analysis demonstrate that the proposed method outperforms the state-of-the-art methods on several public HSI datasets.