Consensus methods to identify a set of potential performance indicators for systems of emergency and urgent care

Objectives: To identify a comprehensive set of indicators to enable Primary Care Trust (PCT) commissioners in England and other NHS decision-makers to monitor the performance of systems of emergency and urgent care for which they are responsible. Methods: Using a combination of Delphi RAND methods in three successive rounds of consultation and nominal group review, we canvassed expert opinion on 70 potential indicators as good measures of system performance. The two Delphi panels consisted of senior clinicians and researchers, and urgent care leads and commissioners in PCTs and Strategic Health Authorities (SHAs). The indicators were formatted into a questionnaire according to whether they were outcome, process, structure, or equity-based measures. Participants scored each indicator on a Likert scale of 1-9 and had the opportunity to consider their scores informed by the group scores and feedback. The questionnaire was refined after each round. To ensure that the indicators rated most highly by the Delphi panels covered all dimensions of performance, the results of the Delphi were reviewed by a nominal group consisting of two researchers and three clinicians from the local health services research network (LHSR). Results: Overall, the process yielded 16 candidate indicators. It also produced a core set of serious, emergency and urgent care-sensitive conditions (defined as conditions whose exacerbations should be managed by a well-performing system without admission to an inpatient bed), for use with the indicators. Conclusions: System-wide measures to monitor performance across multiple services should encourage providers to work for patient benefit in an integrated way. They will also assist commissioners to monitor and improve emergency and urgent care for their local populations. The indicators are now being calculated using routinely available data, and tested for their responsiveness to capture change over time.