Using random rather than fixed effects models in meta-analysis: Implications for situational specificity and validity generalization

Combining statistical information across studies (i.e., meta-analysis) is a standard research tool in applied psychology. The most common meta-analytic approach in applied psychology, the fixed effects approach, assumes that individual studies are homogeneous and are sampled from the same population. This model assumes that sampling error alone explains the majority of observed differences in study effect sizes and its use has lead some to challenge the notion of situational specificity in favor of validity generalization. We critique the fixed effects methodology and propose an advancement-the random effects model (RE) which provides estimates of how between-study differences influence the relationships under study. RE models assume that studies are heterogeneous since they are often conducted by different investigators under different settings. Parameter estimates of both models are compared and evidence in favor of the random effects approach is presented. We argue against use of the fixed effects model because it may lead to misleading conclusions about situational specificity.