Using Electronic Health Records for Surgical Quality Improvement in the Era of Big Data

IMPORTANCE Risk adjustment is an important component of quality assessment in surgical health care. However, data collection places an additional burden on physicians. There is also concern that outcomes can be gamed depending on the information recorded for each patient. OBJECTIVE To determine whether a number of machine-collected data elements could perform as well as a traditional full-risk adjustment model that includes other physician-assessed and physician-recorded data elements. DESIGN, SETTINGS, AND PARTICIPANTS All general surgery patients from the National Surgical Quality Improvement Program database from January 1, 2005, to December 31, 2010, were included. Separate multivariate logistic regressions were performed using either all 66 preoperative risk variables or only 25 objective variables. The area under the receiver operating characteristic curve (AUC) of each regression using objective preoperative risk variables was compared with its corresponding regression with all preoperative variables. Subset analyses were performed among patients who received certain operations. MAIN OUTCOMES AND MEASURES Mortality or any surgical complication captured by the National Surgical Quality Improvement Program, both inpatient and within 30 days postoperatively. RESULTS Data from a total of 745 053 patients were included. More than 15.8% of patients had at least 1 complication and the mortality rate was 2.8%. When examining inpatient mortality, the AUC was 0.9104 with all 66 variables vs 0.8918 with all 25 objective variables. The difference in AUC comparing models with all variables with objective variables ranged from -0.0073 to 0.1944 for mortality and 0.0198 to 0.0687 for complications. In models predicting mortality, the difference in AUC was less than 0.05 among all patients and subsets of patients with abdominal aortic aneurysm repair, pancreatic resection, colectomy, and appendectomy. In models predicting complications, the difference in AUC was less than 0.05 among all patients and subsets of patients with pancreatic resection, laparoscopic cholecystectomy, colectomy, and appendectomy. CONCLUSIONS AND RELEVANCE Rigorous risk-adjusted surgical quality assessment can be performed solely with objective variables. By leveraging data already routinely collected for patient care, this approach allows for wider adoption of quality assessment systems in health care. Identifying data elements that can be automatically collected can make future improvements to surgical outcomes and quality analyses.