Adaptively heterogeneous transfer learning for hyperspectral image classification

Convolutional Neural Network (CNN) has been proved excellent for hyperspectral image(HSI) classification on the basis of large training set. However, the lack of available samples degrades the performance of CNNs in practical HSI classification. Although transfer learning is a typical strategy to address the issue of limited training samples, such methods tend to reload source model's feature extractors directly ignoring their generalization. Furthermore, most existing adaptive transfer learning algorithms pay attention to spectral features of HSI while the obvious difference between RGB image and HSI's spatial features is overlooked. In this paper, the proposed method can solve the mentioned problems to some extent. Firstly, the proposed adaptive heterogeneous transfer (AHT) can measure their transferability and find appropriate solution to transfer knowledge. It computes feature extractors' transferability based on feature maps and influences the learning process of target model through backpropagation, therefore, target model is prone to receive knowledge from feature extractors with high generalization. Secondly, the proposed method lays emphasis on HSI's spatial features. It adopts the methodology of spatial attention and calculates the regularization loss based on spatial features. Experiments were conducted on two widely used HSI datasets. The results demonstrate the method's effectiveness when few samples are available.