Displaying random variation in comparing hospital performance

Introduction: The role of transparency in quality of care is becoming ever more important. Various indicators are used to assess hospital performance. Judging hospitals using rank order takes no account of disturbing factors such as random variation and case-mix differences. The purpose of this article is to compare displays for the influence of random variation on the apparent differences in the quality of care between the Dutch hospitals. Method: The authors analysed the official 2005 data of all 97 hospitals on the following performance indicators: pressure ulcer, cerebro-vascular accident and acute myocardial infarction. The authors calculated CIs of the point estimate and the simulated CIs of the ranks with bootstrap sampling, and visualised the influence of random variation with three modern graphical techniques: forest plot, funnel plot and rank plot. Results: Statistically significant differences between hospitals were found for nearly all performance indicators (p<0.001). However, the CIs in the forest plot revealed that only a small number of hospitals performed significantly better or worse. The funnel plot provides a representation of differences between hospitals compared with a target value and allows for the uncertainty of these differences. The rank plot showed that ranking hospitals was very uncertain. Conclusion: Despite statistically significant differences between hospitals, random variation is a crucial factor that must be taken into account when judging individual hospitals. The funnel plot provides easily interpretable information on hospital performance, including the influence of random variation.