Multi-level difference information replenishment for medical image fusion

Existing image fusion methods always ignore complementary features and saliency from different inputs. To address these limitations, this paper proposes an unsupervised multi-level difference information replenishment fusion network for multimodal medical image fusion (MMIF). Considering some features obliterated between layers, we design a multi-layer feature compensation module in our network to make fused images richer and more complete. Furthermore, we develop a novel fusion strategy to make the result maintain the subjective definition and intuitive features of the original images while adjusting the fused emphasis. On this basis, functional image fusion avoids color distortion by YIN processing. In addition, a hybrid loss is introduced to train our network. L-fid provides the structural similarity for the fidelity term, L-lum is utilized for the luminance maintaining term, and L-sd presents the better gradient for the detail preserving term. Qualitative and quantitative experiments prove the superiority of our method over other state-of-the-art methods.