Attention and transformer complementary fusion network for hyperspectral image spectral reconstruction

Hyperspectral images can be widely used in many fields due to their high information richness. However, costly and complex imaging spectrometers limit its growth. Hyperspectral reconstruction aims to obtain the corresponding hyperspectral image from the multispectral image, to reduce the acquisition cost of hyperspectral images. At present, the related works mainly use methods based on deep learning, and they have achieved good results. However, how to fully extract the global spectral and spatial features of hyperspectral images is still not well solved. To address these issues, we propose an Attention and Transformer Complementary Fusion Network (ATCFNet), which is composed of three modules: Multi-angle Input Image Processing (MIIP), Deep Feature Extraction (DFE) and Hyperspectral Reconstruction (HR) modules. Within the DFE module, an improved Transformer module and a novel Multi-scale Spatial Attention (MSA) module are proposed to extract the global spectral relationship and the spatial features of the hyperspectral images, respectively. Moreover, the MIIP module is proposed to extract the features of input multispectral images more effectively and comprehensively. To verify these, we compare the proposed ATCFNet with other excellent reconstruction methods on three hyperspectral image datasets. The results show that our method achieves the best results in these datasets. Code is publicly available at https://github.com/kidder314/ATCFNet.