MTLSC-Diff: Multitask learning with diffusion models for hyperspectral image super-resolution and classification

Hyperspectral image (HSIs) super-resolution (SR) can improve the spatial resolution of images for subsequent application tasks. In recent years, SR methods based on deep learning have gained widespread attention. However, most of the existing SR methods do not take into account the needs of specific application tasks when designing the network structure. These methods may not be able to efficiently generate high-quality images that satisfy the specific application tasks, leading to degradation of the performance of subsequent application tasks. To solve this problem, we propose a multi-task learning architecture based on the diffusion model, namely MTLSC-Diff. MTLSC-Diff combines the SR network and the classification network in a multitask learning manner on the basis of the diffusion model. MTLSC-Diff achieves mutual guidance of the two tasks by iterating the image super-resolution and classification tasks, thus gradually reconstructing high-quality images and improving classification accuracy. The guided operations for each time step are performed by the specially designed Mutual-Guidance SR-Classification Synergy Module (M-GSCS). M-GSCS refines the multiscale image obtained at the previous time step and uses the predicted high spatial resolution image for classification. Meanwhile, a class-guided SR dynamic refinement strategy (C-GSR) is proposed in M-GSCS, which uses multi-scale classification results to guide target scale images to learn new knowledge to further reconstruct high-quality images. Experimental results on relevant datasets show that our method significantly improves the super-resolution performance as well as the classification performance.