An international inter-rater agreement study in the challenging diagnosis of squamous dysplasia of the oesophagus

Aims: There is a lack of published literature on the diagnostic reproducibility of oesophageal squamous dysplasia using a two-tiered grading system. Methods and results: We identified 75 oesophageal biopsies, which were reviewed by 10 international pathologists twice (1500 total observations) with a > 4-week washout period. Between rounds, a consensus meeting refining diagnostic criteria was held and an atlas was created to provide illustrations. Overall agreement was fair with kappa of 0.35 and 0.32 in round 1 and 2, respectively. Agreement was moderate to fair for negative for dysplasia (ND) (round 1 = 0.41 and round 2 = 0.34) and poor for indefinite for dysplasia (IND) and low-grade dysplasia (LGD) (kappa <0.2). Agreement for high-grade dysplasia (HGD) was good in both rounds (round 1 = 0.64; round 2 = 0.61). In round 1, majority diagnosis (MDx, >= 6 of 10 raters agreeing) was seen in 51 (68%) cases with the majority classified as ND (N = 27) or HGD (N = 18). MDx was rarely achieved for LGD (N = 6). In round 2, MDx was seen in 49 (65%) cases (20 ND, 1 IND, 11 LGD and 17 HGD). Of the 40 cases with MDx for both rounds, 39 were concordant (15 HGD, 6 LGD and 18 ND). Conclusions: Overall agreement among pathologists in diagnosing squamous lesions is fair; however, HGD had good agreement. This study fills an important gap in our knowledge of the inter-rater variability in the diagnosis of oesophageal squamous dysplasia.