Using Prior Information to Plan Appropriately Powered Regression Studies: A Tutorial Using BUCSS

Translational Abstract Statistical power is the probability that a scientific study can detect a purported effect using significance testing methods, assuming that the effect truly exists. Because larger sample sizes are associated with higher statistical power, sample size planning is recommended. There are many types of effect sizes, which measure effect magnitude, and researchers need to determine an appropriate effect size value to input when conducting sample size planning. However, this process can be especially difficult for researchers using linear regression analyses because of the many factors that influence the value and interpretation of the effect size, among other reasons. A potential solution is to use sample effect size information reported in prior research. Unfortunately, the sample effect size should not be used directly in sample size planning. Sample effect sizes from published studies are too high, owing to publication bias, and irrespective of publication, contain uncertainty, because the effect sizes are only estimates of the true, unknown effect size. In this article, an approach, Bias Uncertainty Corrected Sample Size (BUCSS), is demonstrated as a particularly valid method to calculate sample size for regression studies. This tutorial contains a demonstration of BUCSS software in addition to three step-by-step examples of using BUCSS in common regression-based scenarios. Despite increased attention to the role of statistical power in psychological studies, navigating the process of sample size planning for linear regression designs can be challenging. In particular, it can be difficult to decide upon an appropriate value for the effect size, owing to a variety of factors, including the influence of the correlations among the predictors and between the other predictors and the outcome, in addition to the correlation between the particular predictor(s) in question and the outcome, on statistical power. One approach that addresses these concerns is to use available prior sample information but adjust the sample effect size appropriately for publication bias and/or uncertainty. This article motivates a procedure that accomplishes this, Bias Uncertainty Corrected Sample Size (BUCSS), as a valid approach for linear regression, carefully illustrating how BUCSS may be used in practice. To demonstrate the relevant factors influencing BUCSS performance and ensure it performs well in plausible regression contexts, a Monte Carlo simulation is reported. Importantly, the present difficulties in sample size planning for regression are explained, followed by clear illustrations using BUCSS software for a variety of common practical scenarios in regression studies.