Systematic Reviews of Anesthesiologic Interventions Reported as Statistically Significant: Problems with Power, Precision, and Type 1 Error Protection

被引:74
作者
Imberger, Georgina [1 ,2 ]
Gluud, Christian [1 ]
Boylan, John [3 ]
Wetterslev, Jorn [1 ]
机构
[1] Rigshosp, Ctr Clin Intervent Res, Copenhagen Trial Unit, DK-2100 Copenhagen, Denmark
[2] Monash Univ, Dept Anesthesia & Perioperat Med, Melbourne, Vic 3004, Australia
[3] St Vincent Hosp, Dept Anesthesia, Dublin, Ireland
关键词
TRIAL SEQUENTIAL-ANALYSIS; ALLOGENEIC BLOOD-TRANSFUSION; RANDOMIZED CLINICAL-TRIALS; CARDIAC-SURGERY; GENERAL-ANESTHESIA; INFORMATION SIZE; CUMULATIVE METAANALYSIS; MONITORING BOUNDARIES; MYOCARDIAL-INFARCTION; QUALITY;
D O I
10.1213/ANE.0000000000000892
中图分类号
R614 [麻醉学];
学科分类号
100217 ;
摘要
BACKGROUND: The GRADE Working Group assessment of the quality of evidence is being used increasingly to inform clinical decisions and guidelines. The assessment involves explicit consideration of all sources of uncertainty. One of these sources is imprecision or random error. Many published meta-analyses are underpowered and likely to be updated in the future. When data are sparse and there are repeated updates, the risk of random error is increased. Trial Sequential Analysis (TSA) is one of several methodologies that estimates this increased risk (and decreased precision) in meta-analyses. With nominally statistically significant meta-analyses of anesthesiologic interventions, we used TSA to estimate power and imprecision in the context of sparse data and repeated updates. METHODS: We conducted a search to identify all systematic reviews with meta-analyses that investigated an intervention that may be implemented by an anesthesiologist during the perioperative period. We randomly selected 50 meta-analyses that reported a statistically significant dichotomous outcome in their abstract. We applied TSA to these meta-analyses by using 2 main TSA approaches: relative risk reduction 20% and relative risk reduction consistent with the conventional 95% confidence limit closest to null. We calculated the power achieved by each included meta-analysis, by using each TSA approach, and we calculated the proportion that maintained statistical significance when allowing for sparse data and repeated updates. RESULTS: From 11,870 titles, we found 682 systematic reviews that investigated anesthesiologic interventions. In the 50 sampled meta-analyses, the median number of trials included was 8 (interquartile range [IQR], 5-14), the median number of participants was 964 (IQR, 523-1736), and the median number of participants with the outcome was 202 (IQR, 96-443). By using both of our main TSA approaches, only 12% (95% CI, 5%-25%) of the meta-analyses had power 80%, and only 32% (95% CI, 20%-47%) of the meta-analyses preserved the risk of type 1 error <5%. CONCLUSIONS: Most nominally statistically significant meta-analyses of anesthesiologic interventions are underpowered, and many do not maintain their risk of type 1 error <5% if TSA monitoring boundaries are applied. Consideration of the effect of sparse data and repeated updates is needed when assessing the imprecision of meta-analyses of anesthesiologic interventions.
引用
收藏
页码:1611 / 1622
页数:12
相关论文
共 97 条
[81]   Effect of preoperative smoking cessation interventions on postoperative complications and smoking cessation [J].
Thomsen, T. ;
Tonnesen, H. ;
Moller, A. M. .
BRITISH JOURNAL OF SURGERY, 2009, 96 (05) :451-461
[82]  
Thorlund K., 2011, User manual for trial sequential analysis (TSA)
[83]   Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses? [J].
Thorlund, Kristian ;
Devereaux, P. J. ;
Wetterslev, Jorn ;
Guyatt, Gordon ;
Ioannidis, John P. A. ;
Thabane, Lehana ;
Gluud, Lise-Lotte ;
Als-Nielsen, Bodil ;
Gluud, Christian .
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY, 2009, 38 (01) :276-286
[84]   The Number of Patients and Events Required to Limit the Risk of Overestimation of Intervention Effects in Meta-Analysis-A Simulation Study [J].
Thorlund, Kristian ;
Imberger, Georgina ;
Walsh, Michael ;
Chu, Rong ;
Gluud, Christian ;
Wetterslev, Jorn ;
Guyatt, Gordon ;
Devereaux, Philip J. ;
Thabane, Lehana .
PLOS ONE, 2011, 6 (10)
[85]   Single dose oral paracetamol (acetaminophen) for postoperative pain in adults [J].
Toms, Laurence ;
McQuay, Henry J. ;
Derry, Sheena ;
Moore, R. Andrew .
COCHRANE DATABASE OF SYSTEMATIC REVIEWS, 2008, (04)
[86]   The Impact of Study Size on Meta-analyses: Examination of Underpowered Studies in Cochrane Reviews [J].
Turner, Rebecca M. ;
Bird, Sheila M. ;
Higgins, Julian P. T. .
PLOS ONE, 2013, 8 (03)
[87]   Sequential meta-analysis: an efficient decision-making tool [J].
van der Tweel, Ingeborg ;
Bollen, Casper .
CLINICAL TRIALS, 2010, 7 (02) :136-146
[88]   The Efficacy of an Intraoperative Cell Saver During Cardiac Surgery: A Meta-Analysis of Randomized Trials [J].
Wang, Guyan ;
Bainbridge, Daniel ;
Martin, Janet ;
Cheng, Davy .
ANESTHESIA AND ANALGESIA, 2009, 109 (02) :320-330
[89]   Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis [J].
Wetterslev, Jorn ;
Thorlund, Kristian ;
Brok, Jesper ;
Gluud, Christian .
JOURNAL OF CLINICAL EPIDEMIOLOGY, 2008, 61 (01) :64-75
[90]   Estimating required information size by quantifying diversity in random-effects model meta-analyses [J].
Wetterslev, Jorn ;
Thorlund, Kristian ;
Brok, Jesper ;
Gluud, Christian .
BMC MEDICAL RESEARCH METHODOLOGY, 2009, 9