Deep and Bio-Inspired Spiking Neural Networks Based Optimized Multi-modal Neurological Image Fusion Model

Diagnostic information provided by different medical imaging modalities is complementary. Multi-modal medical image fusion seeks to improve visualization, readability, and interpretation of medical anomalies and act as a diagnostic assistance tool. Extracting relevant features from source images with the least manual intervention makes the overall fusion approach more methodical and optimal. In this paper, an optimized medical image fusion model is proposed using Convolutional Neural Networks (CNN) and Coupled Neural P (CNP) systems in the Non-Subsampled Sheartlet Transform (NSST) domain. Low and high-frequency subbands of source images are obtained using NSST. The fusion of low and high-frequency subbands is guided by features extracted by the pre-trained VGG-16 model and optimized CNP system respectively. The optimized CNP system is obtained by optimizing the hyperparameters of the CNP system using the Whale Optimization Algorithm (WOA). The fusion performance is validated by performing several experiments on different MR-SPECT and MR-PET neurological images. Result analysis demonstrates that the proposed fusion framework gives superior visual and parametric performance than existing fusion methods.