Inter-rater reliability in the classification of supraspinatus tendon tears using 3D ultrasound - a question of experience?

Background: Three-dimensional (3D) ultrasound of the shoulder is characterized by a comparable accuracy to two-dimensional (2D) ultrasound. No studies investigating 2D versus 3D inter-rater reliability in the detection of supraspinatus tendon tears taking into account the level of experience of the raters have been carried out so far. Objectives: The aim of this study was to determine the inter-rater reliability in the analysis of 3D ultrasound image sets of the supraspinatus tendon between sonographer with different levels of experience. Patients and methods: Non-interventional, prospective, observational pilot study of 2309 images of 127 adult patients suffering from unilateral shoulder pain. 3D ultrasound image sets were scored by three raters independently. The intra-and inter-rater reliabilities were calculated. Results: There was an excellent intra-rater reliability of rater A in the overall classification of supraspinatus tendon tears (2D vs 3D kappa = 0.892, pairwise reliability 93.81%, 3D scoring round 1 vs 3D scoring round 2 kappa = 0.875, pairwise reliability 92.857%). The inter-rater reliability was only moderate compared to rater B on 3D (kappa = 0.497, pairwise reliability 70.95%) and fair compared to rater C (kappa = 0.238, pairwise reliability 42.38%). Conclusions: The reliability of 3D ultrasound of the supraspinatus tendon depends on the level of experience of the sonographer. Experience in 2D ultrasound does not seem to be sufficient for the analysis of 3D ultrasound imaging sets. Therefore, for a 3D ultrasound analysis new diagnostic criteria have to be established and taught even to experienced 2D sonographers to improve reproducibility.