A transfer learning-based novel fusion convolutional neural network for breast cancer histology classification

To train a convolutional neural network (CNN) from scratch is not suitable for medical image tasks with insufficient data. Benefiting from the transfer learning, the pre-trained CNN model can provide a reliable initial solution for model optimization of medical image classification. A key concern in breast cancer histology classification is that the model should cover the multi-scale features including nuclei-scale, nuclei organization, and structure-scale features. Inspired by these conjectures, we proposed a novel fusion convolutional neural network (FCNN) based on pre-trained VGG19. The FCNN fuses the shallow, intermediate abstract, and abstract layers to approximately cover the multi-scale features. In order to improve the sensitivity of carcinoma classes, the prediction priority is introduced to enable the lesions can be detected as early as possible. Experimental results show that the proposed FCNN can approximately cover the nuclei-scale, nuclei organization, and structure-scale features. Accuracies of 85%, 75%, and 80.56% are achieved in Initial, Extended, and Overall test set, respectively. The source code for this research is available at https://github.com/yxchspring/breasthistolgoy.