Multimodal data integration for predicting progression risk in castration-resistant prostate cancer using deep learning: a multicenter retrospective study

Purpose Patients with advanced prostate cancer (PCa) often develop castration-resistant PCa (CRPC) with poor prognosis. Prognostic information obtained from multiparametric magnetic resonance imaging (mpMRI) and histopathology specimens can be effectively utilized through artificial intelligence (AI) techniques. The objective of this study is to construct an AI-based CRPC progress prediction model by integrating multimodal data.Methods and materials Data from 399 patients diagnosed with PCa at three medical centers between January 2018 and January 2021 were collected retrospectively. We delineated regions of interest (ROIs) from 3 MRI sequences viz, T2WI, DWI, and ADC and utilized a cropping tool to extract the largest section of each ROI. We selected representative pathological hematoxylin and eosin (H&E) slides for deep-learning model training. A joint combined model nomogram was constructed. ROC curves and calibration curves were plotted to assess the predictive performance and goodness of fit of the model. We generated decision curve analysis (DCA) curves and Kaplan-Meier (KM) survival curves to evaluate the clinical net benefit of the model and its association with progression-free survival (PFS).Results The AUC of the machine learning (ML) model was 0.755. The best deep learning (DL) model for radiomics and pathomics was the ResNet-50 model, with an AUC of 0.768 and 0.752, respectively. The nomogram graph showed that DL model contributed the most, and the AUC for the combined model was 0.86. The calibration curves and DCA indicate that the combined model had a good calibration ability and net clinical benefit. The KM curve indicated that the model integrating multimodal data can guide patient prognosis and management strategies.Conclusion The integration of multimodal data effectively improves the prediction of risk for the progression of PCa to CRPC.