Background: Electronic health records (EHRs) may be used to assess quality of care. Objective: To evaluate the accuracy of automated review of EHR data to measure quality of care for outpatients with heart failure. Design: Observational study of quality of care for heart failure comparing automated review of EHR data with automated review followed by manual review of electronic notes for patients with apparent quality deficits (hybrid review). Setting: An academic general internal medicine clinic with several years' experience using a commercial EHR. Patients: 517 adults with a qualifying International Classification of Diseases, Ninth Revision, diagnosis of heart failure in their EHR data and 2 or more clinic visits over the past 18 months. Measurements: Left ventricular ejection fraction (LVEF), prescription of a G-blocker and an angiotensin-converting enzyme (ACE) inhibitor or angiotensin-receptor blocker (ARB) for patients with left ventricular systolic dysfunction (LVEF < 0.40) and prescription of warfarin for patients with comorbid atrial fibrillation. Results: Performance based on automated review of EHR data was similar to that based on hybrid review for assessing LVEF measurement (94.6% vs. 97.3%), prescription of beta-blockers (90.9% vs. 92.8%), and prescription of ACE inhibitors or ARBs (93.9% vs. 98.7%). However, performance based on automated review was lower than that based on hybrid review for prescription of warfarin for atrial fibrillation (70.4% vs. 93.6%), primarily because automated review did not detect documentation of accepted reasons for not prescribing warfarin. Limitations: The findings may not be applicable to other practices and other EHRs. The authors used EHR data to identify eligible patients, so the study may have excluded some patients with heart failure. Patient charts were manually reviewed only if a provider appeared to fail a quality measure on automated review and did not determine the sensitivity and specificity of automated review according to standard definitions. Conclusions: Automated review of EHR data to measure the quality of care of outpatients with heart failure missed many exclusion criteria for medications documented only in providers' notes. As a result, it sometimes underestimated performance on medication-based quality measures.
