Inter-rater reliability of a clinical staging of HIV-associated cognitive impairment

Objective: To determine the inter-rater reliability of a modification of the Memorial Sloan-Kettering (MSK) Staging for HIV-associated cognitive impairment. Methods: Data were abstracted on neurologic, neuropsychological, and functional status on 100 individuals participating at four sites in the Northeast AIDS Dementia (NEAD) Consortium cohort study, a longitudinal study of predictors of cognitive impairment in HIV-infected individuals. Neuropsychological performance was defined 1) based on the neuropsychologist's global impression and 2) solely based on neuropsychological test scores. Raters at each site used the abstracted data to assign an MSK stage to each subject blind to any identifying information. Inter-rater reliability was assessed using kappa statistics. Agreement between computer-generated ratings and site-generated ratings was also assessed. Results: Kappa statistics for pair-wise agreement among the sites regarding MSK stage ranged from 0.70-0.91, representing good to excellent agreement between sites. Agreement between computer-generated ratings and site-generated ratings was in the good to excellent range (0.62-0.79). Conclusions: The authors have modified the MSK rating scale and developed a reliable instrument that can be used in multicenter studies. This instrument will be useful in staging HIV-dementia in future longitudinal studies and will be valuable in increasing accuracy of clinicopathologic studies.