Psychometric Inferences From a Meta-Analysis of Reliability and Internal Consistency Coefficients

A meta-analysis of the reliability of the scores from a specific test, also called reliability generalization, allows the quantitative synthesis of its properties from a set of studies. It is usually assumed that part of the variation in the reliability coefficients is due to some unknown and implicit mechanism that restricts and biases the selection of participants in the studies' samples. Sometimes this variation has been reduced by adjusting the coefficients by a formula associated with range restrictions. We propose a framework in which that variation is included (instead of adjusted) in the models intended to explain the variability and in which parallel analyses of the studies' means and variances are performed. Furthermore, the analysis of the residuals enables inferences to be made about the nature of the variability accounted for by moderator variables. The meta-analysis of the 3 studies' statistics-reliability coefficient, mean, and variance-allows psychometric inferences about the test scores. A numerical example illustrates the proposed framework.