Internal validation of an artificial neural network for prostate biopsy outcome

Objectives: To carry out an internal validation of the retrospectively trained artificial neural network (ANN) 'ProstataClass'. Methods: A prospectively collected database of 393 patients undergoing 8-12 core prostate biopsy was analyzed. Data of these patients were applied to the online available ANN 'ProstataClass' using the Elecsys total prostate-specific antigen (tPSA) and free PSA (fPSA) assays. Beside the internal validation of the ANN 'ProstataClass' an additional ANN (named as ANN internal validation: ANNiv) only using the 393 prospective patient data was evaluated. The new ANN model was constructed with the MATLAB Neural Network Toolbox. Diagnostic accuracy was evaluated by receiver operator characteristic (ROC) curves comparing the areas under the ROC curves (AUC) and specificities at 90% and 95% sensitivity. Results: Within a tPSA range of 1.0-22.8 ng/mL, 229 men (58.3%) had prostate cancer (PCa). tPSA, %fPSA and the number of positive digital rectal examinations (DRE) differed significantly from the cohort of patients of the ANN 'ProstataClass', whereas age and prostate volume were comparable. AUCs for tPSA, %fPSA and the ANN 'ProstataClass' were 0.527, 0.726 and 0.747 (P = 0.085 between %fPSA and ANN). The AUC of the ANNiv (0.754) was significantly better compared with %fPSA (P = 0.021), whereas the AUC of two ANN models built on external cohorts (0.726 and 0.729) showed no differences to %fPSA and the other ANN models. Conclusions: Significant differences of DRE status and %fPSA medians decrease the power of the 'ProstataClass' ANN in the internal validation cohort. The effect of retrospective data evaluation the 'ProstataClass' cohort and prospective fPSA measurement may be responsible for %fPSA differences. All ANN models built with different PSA and fPSA assays performed equally if applied to the two cohorts.