Concordance for Large-Scale Assessments

Interest has grown recently in linking national or regional assessments to international large-scale assessments. However, commonly used equating and linking methods are not defensible for such purposes as they would make unrealistic assumptions such as construct equivalency and error-free measurement, and usually only provide a point to point projection. This paper introduces a new approach for score projection by constructing an enhanced concordance table between two large-scale assessments with one source test and one target test. Specifically, the proposed method employs predictive mean matching method to find a set of donors with the smallest distances to the predicted mean generated by an imputation model on the source test for each concordance level within the identified score range. Both the means and standard deviations of donors' plausible values on the target test are utilized to construct a concordance table between the two tests. This approach not only ensures the score uncertainty due to measurement error and imperfect correlation between tests are appropriately taken into account, but also avoids complex statistical functional forms and linearity assumption. The robustness of the new approach is demonstrated by a linking study to relate a regional assessment to TIMSS and PIRLS international long-standing large-scale assessments, where students take both the source and the target tests. Recommendations for educators and researchers to make inferences and interpret the concordance table are also provided.