The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses

被引：253

作者：

Mustafa, Reem A. ^{[1
]}

Santesso, Nancy ^{[1
]}

Brozek, Jan ^{[1
,2
]}

Akl, Elie A. ^{[1
,3
]}

Walter, Stephen D. ^{[1
]}

Norman, Geoff ^{[1
]}

Kulasegaram, Mahan ^{[1
]}

Christensen, Robin ^{[4
]}

Guyatt, Gordon H. ^{[1
,2
]}

Falck-Ytter, Yngve ^{[5
]}

Chang, Stephanie ^{[6
]}

Murad, Mohammad Hassan ^{[7
]}

Vist, Gunn E. ^{[8
]}

Lasserson, Toby ^{[9
]}

Gartlehner, Gerald ^{[10
]}

Shukla, Vijay ^{[11
]}

Sun, Xin ^{[12
]}

Whittington, Craig ^{[13
]}

Post, Piet N. ^{[14
]}

Lang, Eddy ^{[15
]}

Thaler, Kylie ^{[16
]}

Kunnamo, Ilkka ^{[17
]}

Alenius, Heidi ^{[17
]}

Meerpohl, Joerg J. ^{[18
]}

Alba, Ana C. ^{[1
,19
]}

Nevis, Immaculate F. ^{[1
]}

Gentles, Stephen ^{[1
]}

Ethier, Marie-Chantal ^{[1
,20
]}

Carrasco-Labra, Alonso ^{[1
,21
]}

Khatib, Rasha ^{[1
,22
]}

Nesrallah, Gihad ^{[1
,23
]}

Kroft, Jamie ^{[24
]}

Selk, Amanda ^{[25
]}

Brignardello-Petersen, Romina ^{[1
,21
]}

Schuenemann, Holger J. ^{[1
,2
]}

机构：

[1] McMaster Univ, Dept Clin Epidemiol & Biostat, Hamilton, ON L8S 4K1, Canada

[2] McMaster Univ, Dept Med, Hamilton, ON L8S 4K1, Canada

[3] SUNY Buffalo, Dept Med, Buffalo, NY 14260 USA

[4] Copenhagen Univ Hosp, Musculoskeletal Stat Unit, Parker Inst, Frederiksberg, Denmark

[5] Case Western Reserve Univ, Louis Stokes Cleveland VA Med Ctr, Cleveland, OH 44106 USA

[6] Agcy Healthcare Res & Qual, Rockville, MD 20850 USA

[7] Mayo Clin, Knowledge & Evaluat Res Unit, Rochester, MN USA

[8] Norwegian Knowledge Ctr Hlth Serv, N-0130 Oslo, Norway

[9] Cochrane Editorial Unit, London, England

[10] RTI Int, Durham, NC USA

[11] CADTH, Ottawa, ON K1S 5S8, Canada

[12] Kaiser Permanente Northwest, Ctr Hlth Res, Portland, OR 97227 USA

[13] Natl Collaborating Ctr Mental Hlth, Ctr Outcomes Res & Effectiveness, Res Dept Clin Educ & Hlth Psychol, London, England

[14] Post Voor Zorg, Delft, Netherlands

[15] Univ Calgary, Dept Emergency Med, Calgary, AB 2NT 2T9, Canada

[16] Danube Univ Krems, Austrian Cochrane Branch, Dept Evidence Based Med & Clin Epidemiol, Krems, Austria

[17] Duodecim Med Publicat Ltd, EBM Guidelines, Helsinki, Finland

[18] Univ Med Ctr Freiburg, Inst Med Biometry & Med Informat, German Cochrane Ctr, Freiburg, Germany

[19] Toronto Gen Hosp, Univ Hlth Network, Toronto, ON M5G 2C4, Canada

[20] Hosp Sick Children, Toronto, ON M5G 1X8, Canada

[21] Univ Chile, Evidence Based Dent Unit, Fac Dent, Santiago, Chile

[22] Populat Hlth Res Inst, Hamilton, ON L8L 2X2, Canada

[23] Humber River Reg Hosp, Toronto, ON M5B 1Z2, Canada

[24] Sunnybrook Hlth Sci Ctr, Toronto, ON M4N 3M5, Canada

[25] Womens Coll Hosp, Toronto, ON M5S 1B2, Canada

来源：

JOURNAL OF CLINICAL EPIDEMIOLOGY | 2013年 / 66卷 / 07期

关键词：

GRADE; Inter-rater reliability; Levels of evidence; Evidence-based medicine; Reproducibility; Validation studies; GUIDELINES; STRENGTH;

D O I：

10.1016/j.jclinepi.2013.02.004

中图分类号：

R19 [保健组织与事业（卫生事业管理）];

学科分类号：

摘要：

Objective: We evaluated the inter-rater reliability (IRR) of assessing the quality of evidence (QoE) using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach. Study Design and Setting: On completing two training exercises, participants worked independently as individual raters to assess the QoE of 16 outcomes. After recording their initial impression using a global rating, raters graded the QoE following the GRADE approach. Subsequently, randomly paired raters submitted a consensus rating. Results: The IRR without using the GRADE approach for two individual raters was 0.31 (95% confidence interval [95% CI] = 0.21-0.42) among Health Research Methodology students (n = 10) and 0.27 (95% CI = 0.19-0.37) among the GRADE working group members (n = 15). The corresponding IRR of the GRADE approach in assessing the QoE was significantly higher, that is, 0.66 (95% CI = 0.56-0.75) and 0.72 (95% CI = 0.61-0.79), respectively. The IRR further increased for three (0.80 [95% CI = 0.73-0.86] and 0.74 [95% CI = 0.65-0.81]) or four raters (0.84 [95% CI = 0.78-0.89] and 0.79 [95% CI = 0.71-0.85]). The IRR did not improve when QoE was assessed through a consensus rating. Conclusion: Our findings suggest that trained individuals using the GRADE approach improves reliability in comparison to intuitive judgments about the QoE and that two individual raters can reliably assess the QoE using the GRADE system. (c) 2013 Elsevier Inc. All rights reserved.

引用

页码：736 / 742

页数：7

共 20 条

[1]

[Anonymous], COCHRANE DATABASE SY

[2]

[Anonymous], 2012, COCHRANE DATABASE SY

[3]

[Anonymous], COCHRANE DATABASE SY, DOI [10.1002/14651858.CD003133.pub2, DOI 10.1002/14651858.CD003133.PUB2]

[4] Systems for grading the quality of evidence and the strength of recommendations II:: Pilot study of a new system -: art. no. 25 [J].

Atkins, D ;

Briss, PA ;

Eccles, M ;

Flottorp, S ;

Guyatt, GH ;

Harbour, RT ;

Hill, S ;

Jaeschke, R ;

Liberati, A ;

Magrini, N ;

Mason, J ;

O'Connell, D ;

Oxman, AD ;

Phillips, B ;

Schünemann, H ;

Edejer, TTT ;

Vist, GE ;

Williams, JW .

BMC HEALTH SERVICES RESEARCH, 2005, 5 (1)

[5] GRADE guidelines: 3. Rating the quality of evidence [J].

Balshem, Howard ;

Helfand, Mark ;

Schuenemann, Holger J. ;

Oxman, Andrew D. ;

Kunz, Regina ;

Brozek, Jan ;

Vist, Gunn E. ;

Falck-Ytter, Yngve ;

Meerpohl, Joerg ;

Norris, Susan ;

Guyatt, Gordon H. .

JOURNAL OF CLINICAL EPIDEMIOLOGY, 2011, 64 (04) :401-406

[6]

Fernandes RM, 2010, COCHRANE DB SYST REV, V10

[7] GRADE:: an emerging consensus on rating quality of evidence and strength of recommendations [J].

Guyatt, Gordon H. ;

Oxman, Andrew D. ;

Vist, Gunn E. ;

Kunz, Regina ;

Falck-Ytter, Yngve ;

Alonso-Coello, Pablo ;

Schuenemann, Holger J. .

BRITISH MEDICAL JOURNAL, 2008, 336 (7650) :924-926

[8]

Guyatt GH, 2011, J CLIN EPIDEMIOL, V64, P1311, DOI [10.1016/j.jclinepi.2011.03.017, 10.1016/j.jclinepi.2011.06.004]

[9] GRADE guidelines: 7. Rating the quality of evidence-inconsistency [J].

Guyatt, Gordon H. ;

Oxman, Andrew D. ;

Kunz, Regina ;

Woodcock, James ;

Brozek, Jan ;

Helfand, Mark ;

Alonso-Coello, Pablo ;

Glasziou, Paul ;

Jaeschke, Roman ;

Akl, Elie A. ;

Norris, Susan ;

Vist, Gunn ;

Dahm, Philipp ;

Shukla, Vijay K. ;

Higgins, Julian ;

Falck-Ytter, Yngve ;

Schuenemann, Holger J. .

JOURNAL OF CLINICAL EPIDEMIOLOGY, 2011, 64 (12) :1294-1302

[10] GRADE guidelines: 8. Rating the quality of evidence-indirectness [J].

Guyatt, Gordon H. ;

Oxman, Andrew D. ;

Kunz, Regina ;

Woodcock, James ;

Brozek, Jan ;

Helfand, Mark ;

Alonso-Coello, Pablo ;

Falck-Ytter, Yngve ;

Jaeschke, Roman ;

Vist, Gunn ;

Akl, Elie A. ;

Post, Piet N. ;

Norris, Susan ;

Meerpohl, Joerg ;

Shukla, Vijay K. ;

Nasser, Mona ;

Schuenemann, Holger J. .

JOURNAL OF CLINICAL EPIDEMIOLOGY, 2011, 64 (12) :1303-1310

← 1 2 →