Rater reliability of postnatal urinary tract dilation consensus classification

Background A multidisciplinary urinary tract dilation (UTD) classification system was published in 2014 to standardize definitions and renal/bladder ultrasound image interpretation. Objective To evaluate intra- and inter-rater reliability of this system on postnatal RBUS. Materials and methods Renal/bladder US of 60 infants (<12 months) with urinary tract dilation were anonymized, retrospectively reviewed and scored twice using the UTD classification system by a pediatric urologist and four pediatric radiologists. Exams included supine and prone images of each kidney. Raters recorded the anterior posterior renal pelvis diameter in each position; and when present calyceal dilation (central and peripheral), ureteral dilation, parenchymal and bladder abnormalities. A UTD score was given to each kidney based on these components. Results Intraclass correlation (ICC) of anterior posterior renal pelvis diameter measures was 0.99 (P<0.001). Intra-rater reliability for the anterior posterior renal pelvis diameter of each kidney was high, with ICC >0.95 (P<0.001). Inter-rater kappa values for UTD scores of both kidneys ranged from 0.60 to 0.77 (P<0.001). Intra-rater kappa values for UTD scores of both kidneys ranged from 0.74 to 0.92 (P<0.001). Of the six categories comprising the UTD score, discrepancy between raters was highest for interpretation of central and peripheral calyceal dilation. Conclusion Present inter- and intra-rater reliability findings were similar to those previously reported for grading systems for urinary tract dilation. Across these studies, kappa values are generally lower than the 0.8 cut-off advocated for medical measures. Here, calyceal dilation commonly determined the kidney UTD score and was also the source of greatest discrepancy between raters. Improving consistency of calyceal dilation interpretation may improve UTD consensus score reliability.