Three-dimensional endoanal ultrasonography: intraobserver and interobserver agreement using scoring systems for classification of anal sphincter defects

Objectives To determine the degree of intraobserver and interobserver agreement for an experienced and an inexperienced sonologist using two scoring systems for ultrasonographic assessment of anal sphincter defects. Methods The study sample comprised the datasets of all women aged between 20 and 40 years who had attended our outpatient clinic and undergone a complete three-dimensional (3D) endoanal ultrasound (EAUS) examination in the period from January 2003 to December 2005. The EAUS datasets were assessed twice independently by two sonologists: one with experience of >400 3D EAUS assessments and one inexperienced sonologist who bad performed approximately so assessments before the study. Cases with intraobserver disagreement were resolved, by a third (final) assessment. The final assessment from each observer was used to determine the degree of interobserver agreement. Sphincter defects were classified according to our EAUS defect score and the Starck score. Results EAUS datasets of 55 women were included. Based on first vs. second assessments, intraobserver agreement for the experienced sonologist was good for our EAUS defect score (weighted kappa, 0.75) and the Starck score (weighted kappa, 0.73). Intraobserver agreement for the inexperienced sonologist was moderate for our EAUS defect score (weighted kappa, 0.58) and good for the Starck score (weighted kappa, 0.62). Interobserver agreement was good for both our EAUS defect score (weighted kappa, 0.65) and the Starck score (weighted kappa, 0.74). Conclusions Intraobserver and interobserver agreement was acceptable for both scoring systems. The experienced sonologist obtained a higher degree of intraobserver agreement than did the inexperienced sonologist. Copyright (c) 2009 ISUOG. Publshed by John Wiley & Sons, Ltd.