Regularized spatial-spectral transformer for domain adaptation in hyperspectral image classification

Unsupervised domain adaptation (UDA) is an effective approach for cross-scene hyperspectral image (HSI) classification. Notably, deep learning-based UDA methods have shown potential by leveraging their powerful capability of feature extraction. Most of these methods rely on traditional convolutional neural networks (CNNs), which typically emphasize local feature extraction but lack the capacity to capture global features, thereby limiting their ability to process complex HSI data. Given the ability of capturing long-range dependencies, the transformer demonstrates superior performance than CNNs in many fields, making its employment in UDA a promising approach. However, the original transformer has only spatial attention, which is inadequate for HSI data with spatial and spectral dimensions. Moreover, the limited available labeled HSI data cannot offer the optimization of the transformer model and constrains its capacity. To address these problems, a regularized spatial-spectral transformer for domain adaptation (RSTDA) is proposed. To effectively extract features from HSI data, a spatial-spectral transformer network is designed for HSI data specifically, which contains spatial-spectral attention modules to facilitate the extraction of spatial-spectral features. Also, convolutional layers are introduced at different stages of the transformer as a regulation technique, making it possible to train the transformer model on limited HSI data. Finally, a smooth adversarial training strategy is adopted to decrease the domain discrepancy and improve the generalization of the transformer, thus enhancing the accuracy of the crossscene HSI classification. Experimental results on three datasets demonstrate that RSTDA surpasses the existing state-of-the-art UDA methods for HSI classification.