Brain tumor detection with mRMR-based multimodal fusion of deep learning from MR images using Grad-CAM

The brain tumor is considered a deadly cancer disease that can be seen among people of almost all ages. Early diagnosis and treatment by radiologists are very important in terms of diagnosing an average of 14,000 people with brain tumors per year. Magnetic resonance imaging (MRI) is the best technical expert used for tumor detection via computer-aided systems. However, due to the complex tissue characteristics of a large number of images, manual examination with traditional methods may cause various errors. In this study, a hybrid convolutional neural network (CNN) method based on minimum-redundancy maximum-relevance (mRMR) is proposed, which aims to classify brain tumors over brain MRI with machine learning (ML) algorithms. According to the classification results, DarkNet53, EfficientNet-B0, and DenseNet201 pre-trained CNN architectures, which are three architectures that give the best results as feature extractors, were used as hybrids among 10 different deep learning (DL) architectures. By means of these CNN architectures, the features trained on the features obtained by Gradient-weighted Class Activation Mapping (Grad-CAM) are concatenated. The mRMR method has been used to optimize all concatenated features. Then, the optimized features have classified with SVM, KNN, and Ensemble algorithms. DarkNet53, EfficientNet-B0, and DenseNet201 were used as feature extractors and three different machine learning classifiers were used as classifiers. As a result, the success rate of classification of brain MR images using derived feature vectors has been revealed as 99.6% accuracy with the SVM classifier. According to the experimental results, the use of a combination of feature selection approaches and CNN models helped to successfully classify brain MR images. © The Author(s), under exclusive licence to Springer Nature Switzerland AG 2023.