HYPERSPECTRAL IMAGE-TEXT COUPLING NETWORK FOR HYPERSPECTRAL IMAGE CLASSIFICATION

Deep learning-based approaches have blossomed in the field of hyperspectral image (HSI) classification. However, most of the existing methods focus only on visual information and ignore textual clues. Textual properties can describe the shape, color, and size of ground objects with a wealth of information and are a great complement to visual features. Therefore, the HSI-Text (HSI-T) coupling network is proposed in this study. Specifically, a geocoder is first designed to encode the geospatial information of the image. Moreover, a textual transformer is introduced to extract the attributes of the language. On the basis of the above structures, an HSI-T coupling network is built to explicitly model visual-textual dependencies. Experimental results on benchmark HSI datasets can verify that the proposed approach can gain better classification accuracy and more fine-grained land cover maps.