A Bayesian "Fill-In" Method for Correcting for Publication Bias in Meta-Analysis

Publication bias occurs when the statistical significance or direction of the results between published and unpublished studies differ after controlling for study quality, which threatens the validity of the systematic review and summary of the results on a research topic. Conclusions based on a meta-analysis of published studies without correcting for publication bias are often optimistic and biased toward significance or positivity. We propose a Bayesian fill-in meta-analysis (BALM) method for adjusting publication bias and estimating population effect size that accommodates different assumptions for publication bias. Simulation studies were conducted to examine the performance of BALM and compare it with several commonly used/discussed and recently proposed publication bias correction methods. The simulation results suggested BALM yielded small biases, small RMSE values, and close-to-nominal-level coverage rates in inferring the population effect size and the between-study variance, and outperformed the other examined publication bias correction methods across a wide range of simulation scenarios when the publication bias mechanism is correctly specified. The performance of BALM was relatively sensitive to the assumed publication bias mechanism. Even with a misspecified publication bias mechanism, BALM still outperformed the naive methods without correcting for publication in inferring the overall population effect size. BALM was applied to 2 meta-analysis case studies to illustrate the use of BALM in real life situations. R functions are provided to facilitate the implementation of BALM. Guidelines on how to specify the publication bias mechanisms in BALM and how to report overall effect size estimates are provided. Meta-analysis is widely used in different areas to combine findings from independent studies on a research topic. A frequent threat to the validity of meta-analysis is publication bias. Publication bias occurs when the publication of a study depends not only on the quality of the research but also on the statistical significance or direction of the results. Generally, a study with statistically significant results and/or results that are consistent with the findings of past research is more likely to get published. Without correcting for publication bias, conclusions based on meta-analysis of published studies can be optimistic and biased toward significance or positivity. In this paper, we proposed a Bayesian fill-in meta-analysis (BALM) method for adjusting publication bias and estimating the population effect size in meta-analysis. BALM can accommodate different assumptions for publication bias (e.g., publication bias due to significance and/or directions of study results). Simulation studies with real study sample sizes were conducted to examine the performance of BALM and compare it with several commonly used or recently proposed publication bias correction methods. The simulation results generally showed satisfactory and better inferential properties of BALM in publication bias correction. Two empirical examples were used to illustrate how to use BALM to conduct sensitivity analyses in real life situations. In addition, to facilitate the practical application of the BALM method, we provide R functions balm and balmr for meta-analysis with standardized mean differences and Pearson correlations, respectively.