Joint Cancer Segmentation and PI-RADS Classification on Multiparametric MRI Using MiniSegCaps Network

MRI is the primary imaging approach for diagnosing prostate cancer. Prostate ImagingReporting and Data System (PI-RADS) on multiparametric MRI (mpMRI) provides fundamentalMRI interpretation guidelines but suffers from inter-reader variability. Deep learning networks showgreat promise in automatic lesion segmentation and classification, which help to ease the burdenon radiologists and reduce inter-reader variability. In this study, we proposed a novel multi-branchnetwork, MiniSegCaps, for prostate cancer segmentation and PI-RADS classification on mpMRI.MiniSeg branch outputted the segmentation in conjunction with PI-RADS prediction, guided by theattention map from the CapsuleNet. CapsuleNet branch exploited the relative spatial information ofprostate cancer to anatomical structures, such as the zonal location of the lesion, which also reducedthe sample size requirement in training due to its equivariance properties. In addition, a gatedrecurrent unit (GRU) is adopted to exploit spatial knowledge across slices, improving through-planeconsistency. Based on the clinical reports, we established a prostate mpMRI database from 462 patientspaired with radiologically estimated annotations. MiniSegCaps was trained and evaluated withfivefold cross-validation. On 93 testing cases, our model achieved a 0.712 dice coefficient on lesionsegmentation, 89.18% accuracy, and 92.52% sensitivity on PI-RADS classification (PI-RADS >= 4) inpatient-level evaluation, significantly outperforming existing methods. In addition, a graphical userinterface (GUI) integrated into the clinical workflow can automatically produce diagnosis reportsbased on the results from MiniSegCaps.