Derivation and Validation of the Acute Organ Failure Score to Predict Outcome in Critically Ill Patients: A Cohort Study

Objectives: Prediction models for ICU mortality rely heavily on physiologic variables that may not be available in large retrospective studies. An alternative approach when physiologic variables are absent stratifies mortality risk by acute organ failure classification. Design: Retrospective cohort study. Setting: Two large teaching hospitals in Boston, MA. Subjects: Ninety-two thousand eight hundred eighty-six patients aged 18 years old or older admitted between November 3, 1997, and February 25, 2011, who received critical care. Interventions: None. Measurements and Main Results: The derivation cohort consisted of 35,566 patients from Brigham and Women's Hospital, and the validation cohort comprised 57,320 patients from Massachusetts General Hospital. Acute organ failure was determined for each patient based on International Classification of Diseases, 9th Revision, Clinical Modification code combinations. The main outcome measure was 30-day mortality. A clinical prediction model was created based on a logistic regression model describing the risk of 30-day mortality as a function of age, medical versus surgical patient type, Deyo-Charlson index, sepsis, and type acute organ failure (respiratory, renal, hepatic, hematologic, metabolic, and neurologic) after ICU admission. We computed goodness-of fit statistics and c-statistics as measures of model calibration and 30-day mortality discrimination, respectively. Thirty-day mortality occurred in 5,228 of 35,566 patients (14.7%) assigned to the derivation cohort. The clinical prediction model was predictive for 30-day mortality. The c-statistic for the clinical prediction model was 0.7447 (95% CI, 0.74-0.75) in the derivation cohort and 0.7356 (95% CI, 0.73-0.74) in the validation cohort. For both the derivation and validation cohorts, the Hosmer-Lemeshow chi-square p values indicated good model fit. In a smaller cohort of 444 patients with Acute Physiologic and Chronic Health Evaluation II scores determined, differences in model discrimination of 30-day mortality between the clinical prediction model and Acute Physiologic and Chronic Health Evaluation II were not significant (chi-square = 0.76; p = 0.38). Conclusions: An acute organ failure-based clinical prediction model shows good calibration and discrimination for 30-day mortality in the critically ill. The clinical prediction model compares favorably to Acute Physiologic and Chronic Health Evaluation II score in the prediction of 30-day mortality in the critically ill. This score may be useful for severity of illness risk adjustment in observational studies where physiologic data are unavailable.