A comparison of methods for meta-analysis of a small number of studies with binary outcomes

Meta-analyses often include only a small number of studies (<= 5). Estimating between-study heterogeneity is difficult in this situation. An inaccurate estimation of heterogeneity can result in biased effect estimates and too narrow confidence intervals. The beta-binominal model has shown good statistical properties for meta-analysis of sparse data. We compare the beta-binominal model with different inverse variance random (eg, DerSimonian-Laird, modified Hartung-Knapp, and Paule-Mandel) and fixed effects methods (Mantel-Haenszel and Peto) in a simulation study. The underlying true parameters were obtained from empirical data of actually performed meta-analyses to best mirror real-life situations. We show that valid methods for meta-analysis of a small number of studies are available. In fixed effects situations, the Mantel-Haenszel and Peto methods performed best. In random effects situations, the beta-binominal model performed best for meta-analysis of few studies considering the balance between coverage probability and power. We recommended the beta-binominal model for practical application. If very strong evidence is needed, using the Paule-Mandel heterogeneity variance estimator combined with modified Hartung-Knapp confidence intervals might be useful to confirm the results. Notable most inverse variance random effects models showed unsatisfactory statistical properties also if more studies (10-50) were included in the meta-analysis.