Classification Consistency and Accuracy for Complex Assessments Under the Compound Multinomial Model

For a test that consists of dichotomously scored items, several approaches have been reported in the literature for estimating classification consistency and accuracy indices based on a single administration of a test. Classification consistency and accuracy have not been studied much, however, for "complex'' assessments - for example, those that involve polytomously scored items or mixtures of different types of items. This article describes procedures for estimating various types of single-administration classification consistency and accuracy indices using the multinomial and compound multinomial models for individuals and groups who take a single administration of a complex assessment. An alternative procedure using the bootstrap is also proposed. These procedures are illustrated using real data sets obtained from tests consisting of both polytomous and dichotomous items.