Effect Sizes and Statistical Methods for Meta-Analysis in Higher Education

Quantitative meta-analysis is a very useful, yet underutilized, technique for synthesizing research findings in higher education. Meta-analytic inquiry can be more challenging in higher education than in other fields of study as a result of (a) concerns about the use of regression coefficients as a metric for comparing the magnitude of effects across studies, and (b) the non-independence of observations that occurs when a single study contains multiple effect sizes. This methodological note discusses these two important issues and provides concrete suggestions for addressing them. First, meta-analysis scholars have concluded that standardized regression coefficients, which are often provided in higher education manuscripts, constitute an appropriate metric of effect size. Second, hierarchical linear modeling (HLM) analyses provide an effective method for conducting meta-analytic research while accounting for the non-independence of observations, and HLM is generally superior to other proposed methods that attempt to remedy this same problem. A discussion of how to implement these techniques appropriately is provided.