Bayesian checks on outlying response times in computerized adaptive testing

Bayesian posterior checks for the detection of unexpected response times in computerized adaptive testing are presented. The checks are based on a lognormal model for response times with separate parameters for the slowness of the examinee and the amount of time required by the item. An empirical example in which the response times for the adaptive version of the Arithmetic Reasoning Test in the Armed Services Vocational Aptitude Battery (ASVAB) were studied to detect differential speeedness of the test showed that the detection rates for Bayesian checks outperformed those for classical checks convincingly, but at the cost of slightly higher false-alarm rates. The choice between Bayesian and classical thus basically amounts to the choice of a loss function for the Type 1 and Type 2 errors.