Adaptive Token Mixer for Hyperspectral Image Classification

MLP-like models have shown strong potential in hyperspectral image (HSI) classification. However, their dense connections among all neurons (tokens) lead to large model sizes, high computational costs, and increased risk of overfitting. To address these issues, researchers have proposed sparse connectivity strategies to create more compact MLP models by selecting and mixing only a subset of tokens. However, most token selection rules overlook image patch content, often introducing task-irrelevant tokens with little valuable class distribution information. This problem is particularly severe in HSIs, which contain rich spatial and spectral information. To overcome this, we propose an adaptive token mixer (ATM) to effectively integrate spatial information in HSIs. ATM adaptively learns token positions based on their content, enabling the model to identify relevant tokens and capture global spatial information across the entire spatial domain. In addition, we introduce a cross-shaped convolutional operator (COSTCO) to enhance local spatial feature extraction. The combination of ATM and COSTCO enables comprehensive token mixing by integrating both global and local spatial information. Experimental results show that this proposed adaptive MLP focuses on the most informative, task-relevant regions during decision-making, offering interpretability to help users understand its predictions. Moreover, the adaptive MLP achieves state-of-the-art performance on HSI classification tasks across four publicly available datasets.