Dilated Deep MPFormer Network for Hyperspectral Image Classification

Hyperspectral image (HSI) possesses distinctive advantages in the classification of materials due to its rich spectral information. Convolutional neural network (CNN) and vision transformer (ViT), as mainstream methodologies, have demonstrated significant success. However, they often ignore subtle spectral differences, leading to the inadequate utilization of the spectral information. In this letter, we present a novel feature extraction and classification method, i.e., dilated deep MPFormer network (DDMN), which takes the inherent advantages of CNN and ViT while enhancing the exploitation of spectral information. First, the dilated depthwise separable convolution (DDSC) is proposed to expand the channel dimension, enabling the capture of subtle spectral differences among similar materials. Second, a sequence of improved transformers, i.e., MPFormer, are adopted to effectively extract spatial-spectral features, in which a new multiscale pooling mixer (MPMixer) is designed to replace the attention module in ViT, resulting in reduced parameter numbers and accelerated training speed. Finally, an adaptive weighted fusion module (AWFM) is developed to improve the interaction between specific texture features in shallow layers and abstract semantic features in deep layers. Extensive experiments demonstrate that the proposed DDMN method achieves improvements in OA of 0.8%, 1.04%, and 1.99% when compared to SOTA methods on three HSI datasets of LK, PU, and HS, respectively.