Reliability of NI-RADS criteria in the interpretation of contrast-enhanced magnetic resonance imaging considering the potential role of diffusion-weighted imaging

Objectives To assess inter- and intrareader agreement of the Neck Imaging Reporting and Data System (NI-RADS) used in contrast-enhanced magnetic resonance imaging (MRI) including analysis of diffusion-weighted imaging (DWI), which is currently not part of the NI-RADS criteria. Methods This retrospective study included anonymized surveillance contrast-enhanced MRI datasets of 104 patients treated for different head and neck cancers. Three radiologists experienced in head and neck imaging reported findings for the primary site and the neck using NI-RADS criteria in a first step and evaluated DWI sequences for the primary site in a second step. Thirty randomly selected imaging datasets were again presented to the readers. Kappa statistics and observed agreement (A(o)) were calculated. Results Interreader agreement across all MRI datasets was moderate (kappa(Fleiss) = 0.53) for NI-RADS categories assigned to the primary site, substantial for NI-RADS categories of the neck (kappa(Fleiss) = 0.67), and almost perfect for DWI of the primary site (kappa(Fleiss) = 0.83). Interreader agreement for the primary site was particularly low in cases of cancer recurrence (kappa(Fleiss) = 0.35) and when categories 2a, 2b, and 3 were combined (kappa(Fleiss) = 0.30). Intrareader agreement was considerably lower for NI-RADS categories of the primary site (range A(o) = 53.3-70.0%) than for NI-RADS categories of the neck (range A(o) = 83.3-90.0%) and DWI of the primary site (range A(o) = 93.3-100.0%). Conclusion Interreader agreement of NI-RADS for reporting contrast-enhanced MRI findings is acceptable for the neck but limited for the primary site. Here, DWI has the potential to serve as a reliable additional criterion.