Overtime reliability of medial temporal lobe atrophy rating in a clinical setting

Background: Medial temporal lobe atrophy (MTA) is one of the first magnetic resonance imaging (MRI) signs in patients with Alzheimer's disease (AD) and used as a measure of disease progression. Visual assessment of MTA is easy to perform but the reliability of MTA rating over time has not been studied. Purpose: To investigate what happens to the MTA rating scores if two radiologists rate the same MRI scans six times over a period of 1 year. Material and Methods: One hundred outpatients were included in this study. All patients underwent MRI with a protocol and sequences used for geriatric patients, according to local clinical standards. One neuroradiologist and one general radiologist independently of each other performed retrospective visual assessments of MTA six times, using the same scans, over a period of 1 year. Results: Intra-rater kappa varied between kappa 0.65 and 0.84 for the neuroradiologist and kappa 0.38 and 0.74 for the general radiologist. Intra-rater weighted kappa (w kappa) values showed almost perfect agreement for both investigators (w kappa 0.83-0.94). Inter-rater reliability showed fair to moderate agreement, with the kappa value varying from kappa 0.29 to 0.48 and weighted kappa values ranging from w kappa 0.72 to 0.84. There was a statistically significant difference in rating between the two investigators. Conclusion: Visual assessment of MTA repeated over time has a high grade of reproducibility when performed by an experienced investigator. The reproducibility drops when assessment is rarely performed. Inter-rater reliability is low when two investigators not working together are compared.