MSMT-LCL: Multiscale Spatial-Spectral Masked Transformer With Local Contrastive Learning for Hyperspectral Image Classification

Deep learning plays a crucial role in hyperspectral image (HSI) classification, with the Transformer being highly favored by researchers due to its exceptional ability to model long-range dependencies. However, the Transformer necessitates a substantial amount of labeled training samples to train its numerous parameters, exacerbating the challenge of training an effective HSI classification Transformer model, particularly given the inherent scarcity of HSI data. Therefore, we propose a novel method for HSI classification, termed multiscale spatial-spectral masked Transformer with local contrastive learning (MSMT-LCL). This method consists of two stages: self-supervised pretraining and supervised fine-tuning. Initially, we utilize the multiscale augmented feature mapping module (MAFM) to project original HSI data into two mixed-scale feature maps, which are then separately fed into two masked Transformer branches for reconstruction. To facilitate the model in learning the dependency relationships between central pixel land-cover information and neighboring land cover, we introduce a novel mask strategy based on center-patch. Furthermore, in the pretraining stage, we integrate local contrastive learning (LCL) to enable the model to focus on local center information at varying scales. Upon completion of pretraining, the network undergoes fine-tuning to obtain feature maps at two different scales. Subsequently, we devise a novel adaptive multiscale feature fusion module (AMFM) to adaptively aggregate these two features and produce the final classification results. Extensive experiments on three real datasets demonstrate the superiority of our proposed MSMT-LCL method over several state-of-the-art HSI classification methods.