Dual-Branch Spectral-Spatial Adversarial Representation Learning for Hyperspectral Image Classification With Few Labeled Samples

Recently, deep learning methods, particularly the convolutional neural networks, have been extensively employed for extracting spectral-spatial features in hyperspectral image (HSI) classification tasks, yielding promising results. Conventional methods often use small image patches as input and combine spectral and spatial features with fixed strategies. However, the equal treatment of all pixels within heterogeneous patches can negatively impact feature extraction performance. In this article, we propose a semisupervised dual-branch spectral-spatial adversarial representation learning (SSARL) method for HSI classification. SSARL adaptively assigns attention weights to different pixels and adds a spectral constraint to spatial features. Our approach consists of three main components: 1) a dual-branch framework designed to independently extract spectral and spatial information from pixel and patch samples; 2) a class consistency loss that adaptively combines spectral and spatial classification results, mitigating the adverse effects of heterogeneous patches and enabling appropriate feature selection for various situations; and 3) the deep learning model on the labeled sample size by adding the adversarial representation module and conditional entropy to two branches, reducing the deep learning model's reliance on labeled sample size. Experimental results demonstrate that SSARL outperforms competitive methods on small-sized (0.3%-5%) labeled samples and exhibits superior performance for boundary test pixels.