How do stakeholder groups vary in a Delphi technique about primary mental health care and what factors influence their ratings?

Background: While mental health is a core part of primary care, there are few validated quality measures and little relevant internationally published research. Consensus panel methods are a useful means of developing quality measures where evidence is sparse and/or opinions are diverse. However, little is known about the dynamics of consensus techniques and the factors that influence the judgements and ratings of panels and individual panellists. Objectives: (1) To describe differences in panel ratings on the quality of primary mental health care services by patient, carer, professional and managerial panels within a Delphi procedure; and (2) to explore why different panels and panellists rate quality indicators of primary mental health care differently. Design: Two round postal Delphi technique and exploratory semi-structured interviews. Participants: 115 panellists across 11 panels. Eleven panellists were subsequently interviewed. Results: 87 of 334 indicators (26%) were rated face valid by all 11 panels. There was little disagreement within panel ratings but significant differences between panels. The GP panel rated the least number of indicators valid (n = 138, 41%) and carers the most (n = 304, 91%). The way in which panellists interpreted and conceptualised the indicators and their definition of quality of mental health care affected the way in which participants made their ratings. Conclusions: Stakeholders in primary mental health care have diverse views of quality of care and these differences translate into how they rate quality indicators. Exploratory interviews suggest that ratings are influenced by past experience, expectations, definitions of quality of care, and perceived power relationships between stakeholders.