Exploring variability and sources of measurement error in alcohol expectancy questionnaire reliability coefficients: A meta-analytic reliability generalization study

Objective: Reliability Generalization, a relatively new meta-analytic score reliability technique, was used to examine reliability coefficients for Alcohol Expectancy Questionnaire (AEQ) subscales in published research. Specifically, the present study identified the typical reliability coefficients of scores on AEQ subscales across published studies and examined sources of measurement error across AEQ subscales and studies. Method: Based on literature reviews of PsycINFO and PubMed, a total of 71 studies were analyzed and coded on 10 different dimensions believed to affect score reliability. Results: Only 37% of the studies surveyed reported reliability coefficients for the data in hand. Analysis of these studies revealed that the average score reliability across studies varied considerably in subscales and samples, with the Arousal/Interpersonal Power and Sexual Enhancement subscales evidencing the greatest variability. Gender homogeneity and racial homogeneity were found to be the two most important predictors of the magnitude of score reliability coefficients across subscales and studies. Conclusions: Results of the study indicate that the AEQ tends to generate reliable scores, with some noted exceptions. Because the Arousal/ Interpersonal Power and Sexual Enhancement subscales tend to generate unacceptably low reliability coefficients and because the AEQ holds promise in furthering prevention and treatment outcome research, it is suggested that future research with the AEQ explore instrument factor structure and item consistency. Because reliability estimates can influence substantive statistical analyses, it is recommended that authors using the AEQ report reliability information in their published work.