A multi-view deep learning model for pathology image diagnosis

Pathology image diagnosis plays a critical role in cancer diagnosis and treatment. However, due to the serious lack of experienced pathologists, computer-aided pathological diagnosis has become extremely important. In addition, while machine learning technologies have been successfully and widely used in other medical fields, there is still a lack of computer intervention in the basic process of diagnosis in pathology images. This paper proposes a multi-view deep learning model for pathology image diagnosis (named MvPID), which combines image features and multi-view deep learning networks. Specifically, first, the whole slide image is segmented into different non-overlapping sub-slices. Then, we extract different image features from sub-slices as different views for multi-view learning. Subsequently, we propose to use the view-specific deep Gaussian processes to extract the unique information of different views and the view-common autoencoder (AE) network to integrate the information of different views into a common representation. The common representation is put into the downstream classifier to realize automatic pathological diagnosis. The experimental results on real pathological data show that the proposed approach is effective. The best classification performance far exceeds the diagnosis accuracy of pathologists, which proves the application potential of the proposed MvPID.