Interobserver agreement and the impact of mentorship on the diagnosis of inflammatory bowel disease-associated dysplasia among subspecialist gastrointestinal pathologists

The diagnosis of inflammatory bowel disease (IBD)-associated dysplasia is challenging, and past studies have demonstrated considerable interobserver variability in such diagnoses. This study aimed to assess interobserver agreement in IBD dysplasia diagnoses among subspecialty GI pathologists and to explore the impact of mentorship on diagnostic variability. Twelve GI pathologist mentees and 7 GI pathologist mentors reviewed 163 digitized slides. Participants rendered a diagnosis of negative for dysplasia, indefinite for dysplasia, low-grade dysplasia, or high-grade dysplasia and provided a confidence level for each case. Interobserver agreement and reliability were assessed using Cohen's and Fleiss' kappa (kappa) statistics and intraclass correlation coefficient (ICC) analysis. The overall kappa coefficient was 0.42 (95% CI: 0.38-0.46). The overall ICC was 0.67 (95% CI: 0.62-0.72). Kappa coefficients ranged from 0.31 to 0.49 for mentor/mentee pairs and from 0.34 to 0.55 for pairs of mentees of the same mentor. The combined kappa coefficient was 0.44 (95% CI: 0.39-0.48) for all mentees and 0.39 (95% CI: 0.34-0.43) for all mentors. Common features in low agreement cases included mucosal atrophy, areas of stark contrast, serrations, decreased goblet cells, absent surface epithelium, and poor orientation. Participants were confident in most diagnoses, and increased confidence levels generally correlated with higher interobserver agreement. Interobserver agreement among subspecialist GI pathologists in this curated cohort of IBD dysplasia cases was fair to moderate. Mentorship during GI pathology fellowship does not appear to be a significant factor contributing to interobserver variability, but increased experience also does not seem to improve interobserver agreement.