Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists

Background There is increasing research in using segmentation of prostate cancer to create a digital 3D model from magnetic resonance imaging (MRI) scans for purposes of education or surgical planning. However, the variation in segmentation of prostate cancer among users and potential inaccuracy has not been studied. Methods Four consultant radiologists, four consultant urologists, four urology trainees, and four nonclinician segmentation scientists were asked to segment a single slice of a lateral T3 prostate tumor on MRI ("Prostate 1"), an anterior zone prostate tumor MRI ("Prostate 2"), and a kidney tumor computed tomography (CT) scan ("Kidney"). Time taken and self-rated subjective accuracy out of a maximum score of 10 were recorded. Root mean square error, Dice coefficient, Matthews correlation coefficient, Jaccard index, specificity, and sensitivity were calculated using the radiologists as the ground truth. Results There was high variance among the radiologists in segmentation of Prostate 1 and 2 tumors with mean Dice coefficients of 0.81 and 0.58, respectively, compared to 0.96 for the kidney tumor. Urologists and urology trainees had similar accuracy, while nonclinicians had the lowest accuracy scores for Prostate 1 and 2 tumors (0.60 and 0.47) but similar for kidney tumor (0.95). Mean sensitivity in Prostate 1 (0.63) and Prostate 2 (0.61) was lower than specificity (0.92 and 0.93) suggesting under-segmentation of tumors in the non-radiologist groups. Participants spent less time on the kidney tumor segmentation and self-rated accuracy was higher than both prostate tumors. Conclusion Segmentation of prostate cancers is more difficult than other anatomy such as kidney tumors. Less experienced participants appear to under-segment models and underestimate the size of prostate tumors. Segmentation of prostate cancer is highly variable even among radiologists, and 3D modeling for clinical use must be performed with caution. Further work to develop a methodology to maximize segmentation accuracy is needed.