(How) Are quality indicators for measuring and appraising the quality of healthcare derived from evidence-based clinical practice guidelines? A review

Background: Evidence-based and formally consensus-based clinical practice guidelines (CPGs) offer potential for the development of quality indicators (QIs). Although QIs are recommended as part of some CPGs, there is no accepted gold standard for the specific development process of guideline-based QIs. The purpose of this review, which is embedded in a mixed-methods research project, was to analyze the current state of methodological approaches for QI development in German CPGs to derive insights for the development of a national evidence-guided and consensus-based standard for guideline-based development of QIs. Methods: In order to identify valid CPGs containing recommendations for QIs, a search was carried out (July 31, 2016) via the guideline database of the German Association of the Scientific Medical Societies (AWMF). Based on a stratified random sample per guideline program (guidelines published by medical societies, National Program for Disease Management Guidelines (DMG), and the German Guideline Program in Oncology [GGPO]), 11 CPGs were selected. With regard to QIs, the specific development methodology, indications on their psychometric properties and how the quality of care should be examined by recommended QIs were extracted and compared by using the guideline documents. Results: In 35 of the 109 (16/85 medical societies, 4/8 DMG, 15/16 GGPO) (32 %) valid CPGs, a total of 372 QIs were recommended. Based on 11 randomly selected guidelines (5 published by medical societies, 1 DMG, 5 GGPO; a total of 109 QIs), the QI development methodology was inconsistent in all five medical societies guidelines (including QI presentation, usage and selection of guideline recommendations for QI derivation) compared to DMG and GGPO. Based on all 109 QIs, 2 (2 %) were presented as a quantitative measure with a reference range, and quality objectives were formulated for 17 (16 %). There was no guideline explicitly reporting about the results of a pilot study or data-based analysis of the psychometric properties of the recommended QIs. The GGPO guideline documents were the only ones providing information on the assessment of the quality of care based on recommended QIs. Discussion: The usage of the QI manuals of the DMG and GGPO leads to a largely standardized development of guideline-based QIs. In the CPGs of the medical societies - if at all - QIs are developed inconsistently and mostly unsystematically. Due to largely missing reference ranges and quality objectives, the identified QIs cannot yet be used to transparently identify potential quality deficits in health care. This requires results of pilot studies and further development of guideline-based QI. Conclusions: A standard for QI development is needed for German guideline authors to seize the opportunity and develop clinically relevant, widely accepted and evidence-based QIs in the guideline development process. In addition, it must be ensured that appropriate structures are used or set up in order to be able to apply the recommended QIs in the German healthcare system.