Evaluation of a consumer-oriented internet health care report card - The risk of quality ratings based on mortality data

Context Healthcare "report cards" have attracted significant consumer interest, particularly publicly available Internet health care quality rating systems. However, the ability of these ratings to discriminate between hospitals is not known. Objective To determine whether hospital ratings for acute myocardial infarction (AN) mortality from a prominent Internet hospital rating system accurately discriminate between hospitals' performance based on process of care and outcomes. Design, Setting, and Patients Data from the Cooperative Cardiovascular Project, a retrospective systematic medical record review of 141914 Medicare fee-for-service beneficiaries 65 years or older hospitalized with AM[ at 3363 US acute care hospitals during a 4- to 8-month period between January 1994 and February 1996 were compared with ratings obtained from HealthGrades.com (1-star worse outcomes than predicted, 5-star: better outcomes than predicted) based on 1994-1997 Medicare data. Main Outcome Measures Quality indicators of AN care, including use of acute reperfusion therapy, aspirin, beta-blockers, angiotensin-converting enzyme inhibitors; 30-day mortality. Results Patients treated at higher-rated hospitals were significantly more likely to receive aspirin (admission: 75.4% 5-star vs 66.4% 1-star, P for trend =.001; discharge: 79.7% 5-star vs 68.0% 1-star, P=.001) and beta-blockers (admission: 54.8% 5-star vs 35.7% 1-star, P=.001; discharge: 63.3% 5-star vs 52.1% 1-star, P=.001), but not angiotensin-converting enzyme inhibitors (59.6% 5-star vs 57.4% 1-star, P=.40). Acute reperfusion therapy rates were highest for patients treated at 2-star hospitals (60.6%) and lowest for 5-star hospitals (53.6% 5-star, P=.008). Risk-standardized 30-day mortality rates were lower for patients treated at higher-rated than lower-rated hospitals (21.9% 1-star vs 15.9% 5-star, P=.001). However, there was marked heterogeneity within rating groups and substantial overlap of individual hospitals across rating strata for mortality and process of care; only 3.1% of comparisons between 1-star and 5-star hospitals had statistically lower risk-standardized 30-day mortality rates in 5-star hospitals. Similar findings were observed in comparisons of 30-day mortality rates between individual hospitals in all other rating groups and when comparisons were restricted to hospitals with a minimum of 30 cases during the study period. Conclusion Hospital ratings published by a prominent Internet health care quality rating system identified groups of hospitals that, in the aggregate, differed in their quality of care and outcomes. However, the ratings poorly discriminated between any 2 individual hospitals' process of care or mortality rates during the study period. Limitations in discrimination may undermine the value of health care quality ratings for patients or payers and may lead to misperceptions of hospitals' performance.