Toward Universal Representation Learning for Multidomain Hyperspectral Image Classification

Deep learning-based methods have greatly improved the performance of hyperspectral image (HSI) classification over the past several years. Nevertheless, current deep learning methods require training and deploying an independent model for each hyperspectral data domain. Representations learned for one data domain can hardly be generalized to other data domains, so multiple models are needed in real-world applications when data from multiple domains are involved. In this article, we design a single neural network that learns universal representations simultaneously from multiple hyperspectral remote sensing data domains. The universal convolutional neural network (CNN) adapts its behavior to different hyperspectral datasets. The majority of parameters of the network are shared to learn common knowledge from multiple datasets. A small number of domain-specific parameters are assigned to handle the domain shift. In addition, we propose a two-step training strategy to fully utilize the capacity of the universal network. Experiments conducted on seven HSI datasets demonstrate that the proposed universal network outperforms multiple individual specialized single-domain networks.