Quantifying Uncertainty in the Meta-Analytic Lower Bound Estimate

In meta-analyses, it is customary to compute a confidence interval for the overall mean effect ((rho) over bar or (delta) over bar), but not for the underlying standard deviation (tau) or the lower bound of the credibility value (90%CV), even though the latter entities are often as important to the interpretation as is the overall mean. We introduce 2 methods of computing confidence intervals for the lower bound (Lawless and bootstrap). We compare both methods using 3 lower bound estimators (Schmidt-Hunter, Schmidt-Hunter with k correction, and Morris/Hedges, labeled HOVr/HOVd) in 2 Monte Carlo studies (1 for correlations and 1 for standardized mean differences) and illustrate their application to published meta-analyses. For correlations, we found that HOVr bootstrap confidence intervals yielded coverage greater than 90% across a wide variety of conditions provided that there were at least 25 studies. For the standardized mean difference, all 3 methods produced acceptable results using the bootstrap for the lower bound confidence interval provided that there were at least 20 studies with an average sample size of at least 50. When the number of studies was small (k = 8 or 10), coverage was less than 90% and confidence intervals were very wide. Even with larger numbers of studies, if there was indirect range restriction coupled with a small underlying between-studies variance, the between-studies variance was poorly estimated and coverage of the lower bound suffered. We provide software to allow meta-analysts to compute bootstrap confidence intervals for the estimators described in the article.