Multimodal classification of breast cancer using feature level fusion of mammogram and ultrasound images in machine learning paradigm

The manual examination of Breast Cancer (BC) images for disease detection is prone to error, is time-consuming, and has low accuracy. The Computer-Aided Detection (CAD) system can overcome the above-mentioned limitations, however, the existing single modality studies have limited clinical applications because the Radiologists mostly make use of both ultrasound with a corresponding mammogram and vice-versa for final diagnosis. This paper presents a novel semi-automated multimodal classification system of breast tumors by combining features from both mammogram and ultrasound images. Forty-two grayscale features were extracted followed by statistical significance analysis to determine the most relevant features and classify the tumors as benign or malignant. A new database consisting of 43 mammograms and 43 ultrasounds is used in the experiments. The size of the dataset is increased further by applying several data augmentation techniques. Also, filtering is applied to deal with artifacts and noise present in the images which may affect the performance accuracy of a model. The performance of the proposed multimodal approach is evaluated using different machine learning classifiers under a tenfold data division protocol. The results demonstrate that combined mammogram and ultrasound achieve a high classification accuracy of 98.84% with cubic Support Vector Machine (SVM) for the proposed multimodal CAD system compared to mammography and ultrasound alone which achieves an accuracy of 93.41% and 91.67% respectively. Our results support the current clinical practice of utilizing mammogram and ultrasound together for improved BC screening.