A super-resolution fusion video imaging spectrometer based on single-pixel camera

Multispectral cameras collect image data through more spectral channels, thereby providing higher level of detail spectral information. The high resolution of large-scale multispectral data in the spectrum, space and time dimensions has become a major challenge in the design of imaging spectrometers. The idea of single-pixel imaging (SPI) has the potential to balance many indicators of imaging spectrometers due to its inherent high sensitivity and low cost. For the limitation of the reconstruction mechanism, SPI is more difficult to image transformed scenes, and is more significant for spectral imaging of transformed scenes. This paper proposes a dual optical path spectral imaging system based on SPI, SPFS (Single pixel fusion spectrometer), which greatly reduces the sampling time by reducing the SPI imaging resolution. SPFS does not require spectral unmixing work since the observed vector does not directly contain spatial information, plus the data of different bands are directly completed by the linear array spectrometer. The high spatial resolution image acquired by the RGB camera is used to fuse with the low spatial resolution spectral image reconstructed by SPI. In addition, a lightweight deep learning algorithm is used to cooperate with this system to process image fusion. Experimental verification and theoretical analysis show that the working frame rate of this scheme is able to reach at least 24 Hz, and it has the potential for special optimization under different wavelength ranges and a higher frame rate improvement space.