Automated Medical Diagnosis System Based on Multi-modality Image Fusion and Deep Learning

Multi-modality medical image fusion aims at integrating information from medical images with different modalities to aid in the diagnosis process. Most research work in this area ends with the fusion stage only. This paper, on the contrary, tries to present a complete diagnosis system based on multi-modality image fusion. This system works on MR and CT images. It begins with the registration step using Scale-Invariant Feature Transform registration algorithm. After that, histogram matching is performed to allow accurate fusion of the medical images. Two methods of the fusion are utilized and compared, wavelet and curvelet fusion. An interpolation stage is included to enhance the resolution of the obtained image after fusion. Finally, a deep learning approach is adopted for classification of images as normal or abnormal. Simulation results reveal good success of the proposed automated diagnosis system based on the fusion and interpolation results.