Predicting Metastasis Risk in Pancreatic Neuroendocrine Tumors Using Deep Learning Image Analysis

Background The prognosis of patients with pancreatic neuroendocrine tumors (PanNET), the second most common type of pancreatic cancer, varies significantly, and up to 15% of patients develop metastasis. Although certain morphological characteristics of PanNETs have been associated with patient outcome, there are no available morphology-based prognostic markers. Given that current clinical histopathology markers are unable to identify high-risk PanNET patients, the development of accurate prognostic biomarkers is needed. Here, we describe a novel machine learning, multiclassification pipeline to predict the risk of metastasis using morphological information from whole tissue slides. Methods Digital images from surgically resected tissues from 89 PanNET patients were used. Pathologist-annotated regions were extracted to train a convolutional neural network (CNN) to identify tiles consisting of PanNET, stroma, normal pancreas parenchyma, and fat. Computationally annotated cancer or stroma tiles and patient metastasis status were used to train CNN to calculate a region based metastatic risk score. Aggregation of the metastatic probability scores across the slide was performed to predict the risk of metastasis. Results The ability of CNN to discriminate different tissues was high (per-tile accuracy >95%; whole slide cancer regions Jaccard index = 79%). Cancer and stromal tiles with high evaluated probability provided F1 scores of 0.82 and 0.69, respectively, when we compared tissues from patients who developed metastasis and those who did not. The final model identified low-risk (n = 76) and high-risk (n = 13) patients, as well as predicted metastasis-free survival (hazard ratio: 4.71) after adjusting for common clinicopathological variables, especially in grade I/II patients. Conclusion Using slides from surgically resected PanNETs, our novel, multiclassification, deep learning pipeline was able to predict the risk of metastasis in PanNET patients. Our results suggest the presence of prognostic morphological patterns in PanNET tissues, and that these patterns may help guide clinical decision making.