Enhancing Remote Sensing Scene Classification With Hy-MSDA: A Hybrid CNN-Transformer for Multisource Domain Adaptation

Multisource unsupervised domain adaptation (MUDA) has demonstrated its effectiveness in improving model performance for remote sensing (RS) scene classification, particularly in cases where the target-domain lacks labeled data. However, most current methods based on convolutional neural networks (CNNs) or Transformers fail to fully exploit the features within each source domain, which benefits classification accuracy. Additionally, these methods frequently overlook the varying levels of similarity between domains, limiting the potential of MUDA. To alleviate this limitation, we propose Hy-MSDA, a hybrid CNN-Transformer with consistent learning and dynamic weighting for MUDA, which fully explores and utilizes valuable information from multiple sources at both the feature and decision levels. To achieve a better alignment of categories among domains, the consistency learning module aims to learn domain-invariant features, maintaining consistency in global high-level features and rich semantic information at the feature level. At the decision level, the dynamic weighting strategy balances the contribution of each source domain by considering their varying importance levels. These two components are closely interconnected and mutually reinforce each other, resulting in improved cross-domain collaboration of Hy-MSDA for domain adaptation. Experimental results on the RS classification datasets clearly demonstrate that Hy-MSDA outperforms the state-of-the-art methods, achieving a significant enhancement of 2%-7% in classification accuracy. Moreover, experiments on RS segmentation datasets reveal that Hy-MSDA performs comparably to supervised methods, further validating the effectiveness of Hy-MSDA in segmentation tasks. The source code is available at https://github.com/phaeton2017/Hy-MSDA.