Inter-rater reliability of retrograde urethrograms

PurposeReliability of pre-operative testing is important for adequate surgical planning. For urethral stricture disease, preoperative planning frequently includes retrograde urethrogram (RUG). The radiographic interpretation of RUGs is often done by urologists themselves. We aimed to evaluate the reliability of RUG interpretation by urologists at our institution.MethodsWe examined the RUGs of 193 patients. These were deidentified and interpreted by three urologists, two general urologists and one reconstructive urologist. These interpretations were compared in 2 ways. Each of the general urologists was compared to the "gold standard" reconstructive urologist interpretation, and the general urologists were additionally compared to each other. We used intraclass correlation coefficient (ICC) for numerical variables and Fleiss' Kappa or Cohen's Kappa statistic (kappa) for categorical variables to rate inter-interpreter reliability and agreement among interpretations with regards to the quantitative variables of stricture length and caliber.ResultsLevel of agreement ranged from poor to moderate across all variables interpreted. Comparing general urologists to the gold standard yielded no better than moderate agreement, with the majority being poor to fair. Similarly, agreement amongst the general urologists did not reach above moderate, with the majority being poor to slight.ConclusionTo our knowledge, this is the first analysis of inter-rater reliability of RUGs among practicing urologists. Our analysis showed clinically unacceptable reliability with regards to stricture length, location, caliber, and indicated procedures. This study suggests a need for standardized interpretation of RUGs and poses an opportunity for actionable improvement in management of strictures.