CentralFormer: Centralized Spectral-Spatial Transformer for Hyperspectral Image Classification With Adaptive Relevance Estimation and Circular Pooling

Classification of hyperspectral image (HSI) is a hotspot in the field of remote sensing. Recent deep learning (DL)-based approaches, especially for the transformer architectures, have been investigated to extract the deep spectral-spatial features. However, the ability of these approaches to efficiently represent the crucial attention patterns and distinguishing features suffers from the neglect of the relevant areas, including the center pixel and high computational complexity; thereby, their classification performances still need to be improved. This article proposes a centralized spectral-spatial transformer (CentralFormer), which contains the central encoder, the adaptive relevance estimation (ARE) module, and the cross-encoder relevance fusion (CERF) module, for HSI classification. To recognize the relevant areas, the ARE modules access both spectral and spatial associations between the center pixel and its neighborhoods flexibly. By focusing on these areas and emphasizing them during attention inference, the central encoders can extract the key attention modes and discriminating features effectively. Moreover, the CERF modules are deployed to prevent the reliability of the relevance map from being harmed by the feature deviation between encoders. To handle the high computational occupancy, a novel circular pooling (CP) strategy reduces the circles and bands of features. Unlike regular pooling methods, it can well improve the relevant characteristics for subsequent encoders. By integrating these techniques, the CentralFormer model can represent the discriminating spectral-spatial features efficiently for HSI classification. Experimental results on four classic HSI datasets reveal that the proposed CentralFormer model outperforms the state-of-the-arts in terms of both classification accuracy and computational efficiency. The source code is available at https://github.com/ningyang-li/CentralFormer.