The Impact of Model Misspecification with Multidimensional Test Data

In this study data were simulated for 5000 examinees on a thirty-item two-dimensional test, using a compensatory MIRT model. Various combinations of simple and complex structure items were examined. Specifically, the numbers of simple structure items on the tests were gradually decreased from 24 to 6, in multiples of six, while simultaneously increasing the number of complex items by the same number of items. In one scenario, the simple structure items were simulated to measure both dimensions equally; in a second scenario, the simple structured items were simulated to measure only the first dimension. The current investigation also varied the correlation between dimensions and the ability distributions on the first and second dimensions. RMSE was used to determine the impact of model misspecification and the results of a unidimensional simulated and scaled test were used for comparison purposes. Results indicated that the underlying structure of multidimensional tests did have an impact on estimation error. However, in some instances fitting a unidimensional model to multidimensional data resulted in estimation error that was not very dissimilar from what was obtained when fitting a unidimensional model to unidimensional data.