Type I Error Rates and Power of Several Versions of Scaled Chi-Square Difference Tests in Investigations of Measurement Invariance

A Monte Carlo simulation study was conducted to investigate Type I error rates and power of several corrections for nonnormality to the normal theory chi-square difference test in the context of evaluating measurement invariance via structural equation modeling. Studied statistics include the uncorrected difference test, DML, Satorra and Bentler's (2001) original correction, DSB1, Satorra and Bentler's (2010) strictly positive correction, DSB10, and a hybrid procedure, DSBH (Asparouhov & Muthen, 2013). Multiple-group data were generated from confirmatory factor analytic population models invariant on all parameters, or lacking invariance on residual variances, indicator intercepts, or factor loadings. Conditions varied in terms of the number of indicators associated with each factor in the population model, the location of noninvariance (if any), sample size, sample size ratio in the 2 groups, and nature of nonnormality. Type I error rates and power of corrected statistics were evaluated for a series of 4 nested invariance models. Overall, the strictly positive correction, DSB10, is the best and most consistently performing statistic, as it was found to be much less sensitive than the original correction, DSB1, to model size and sample evenness. Translational Abstract Measurement invariance is the property of psychometric instruments (e. g., personality inventories, intelligence tests) indicating equivalent functioning when applied to members of different subpopulations (e. g., nationality, sex, socioeconomic status), or across time-points. When invariance does not hold for a measure, one cannot say that the same psychological construct is being measured in all populations or at all time-points, thus rendering the scores of individuals from different populations incomparable and measured change from 1 time-point to the next meaningless. The present study uses simulated data to compare the performance of several test statistics in terms of their ability to identify measurement invariance when it is present, and to detect violations of measurement invariance when they are present. Overall, it was found that a statistic proposed by Satorra and Bentler (2010), referred to as the strictly positive scaled chi-square difference test, or DSB10, is the best performing statistic when testing measurement invariance. It is recommended that practitioners use this statistic, as opposed to other more popular statistics, when evaluating measurement invariance.