Do patient-reported outcomes offer a more sensitive method for comparing the outcomes of consultants than mortality? A multilevel analysis of routine data

Background Patient-reported outcome measures (PROM) might be better for comparing consultant surgeons' outcomes than mortality. Objectives To describe variation in outcomes between consultants, compare the number of outlying consultants according to different measures, explore the effect that the hospital in which a consultant works has on their outcomes and determine the scope for improving outcomes by reducing variation between consultants. Method Consultants performing hip replacement (n=948), knee replacement (1130) and hernia repair (974) in National Health Service hospitals in England in 2009-2012; disease-specific and generic PROMs and complications; fixed-effects and multilevel models to assess consultant outcomes, were all compared. Influence of patient factors and hospital factors was assessed. Results Fixed-effects models showed that most consultants are 'as or better than expected'. However, unlike with mortality, some consultants are more than three SDs 'worse than expected' according to disease-specific PROMs (2.4% for hip and 1.2% for knee replacement), generic PROMs (1.2% and 1.0%) and incidence of complications (1.8% and 0.8%). The proportion of consultants worse than expected is less with random-effects models. Controlling for hospital factors reduced the proportion further. After controlling for known patient characteristics, consultants and hospitals contribute little towards variation in patient outcomes. Conclusions PROMs offer a more appropriate and sensitive method for comparing consultants' outcomes. The influence of hospitals must be considered to ensure comparisons are meaningful. Improvements will be achieved by shifting the distribution of consultants rather than by reducing variation between them.