Validation of the International Classification of Diseases, Tenth Revision Code for the National Institutes of Health Stroke Scale Score

Background:Administrative data can be useful for stroke research but have historically lacked data on stroke severity. Hospitals increasingly report the National Institutes of Health Stroke Scale (NIHSS) score using an International Classification of Diseases, Tenth Revision (ICD-10) diagnosis code, but this code's validity remains unclear. Methods:We examined the concordance of ICD-10 NIHSS scores versus NIHSS scores recorded in CAESAR (Cornell Acute Stroke Academic Registry). We included all patients with acute ischemic stroke from October 1, 2015, when US hospitals transitioned to ICD-10, through 2018, the latest year in our registry. The NIHSS score (range, 0-42) recorded in our registry served as the reference gold standard. ICD-10 NIHSS scores were derived from hospital discharge diagnosis code R29.7xx, with the latter 2 digits representing the NIHSS score. Multiple logistic regression was used to explore factors associated with availability of ICD-10 NIHSS scores. We used ANOVA to examine the proportion of variation (R-2) in the true (registry) NIHSS score that was explained by the ICD-10 NIHSS score. Results:Among 1357 patients, 395 (29.1%) had an ICD-10 NIHSS score recorded. This proportion increased from 0% in 2015 to 46.5% in 2018. In a logistic regression model, only higher registry NIHSS score (odds ratio per point, 1.05 [95% CI, 1.03-1.07]) and cardioembolic stroke (odds ratio, 1.4 [95% CI, 1.0-2.0]) were associated with availability of the ICD-10 NIHSS score. In an ANOVA model, the ICD-10 NIHSS score explained almost all the variation in the registry NIHSS score (R-2=0.88). Fewer than 10% of patients had a large discordance (>= 4 points) between their ICD-10 and registry NIHSS scores. Conclusions:When present, ICD-10 codes representing NIHSS scores had excellent agreement with NIHSS scores recorded in our stroke registry. However, ICD-10 NIHSS scores were often missing, especially in less severe strokes, limiting the reliability of these codes for risk adjustment.