A new look at the influence of guessing on the reliability of multiple-choice tests

Previous studies have established that chance success due to guessing contributes to error variance and diminishes the reliability of multiple-choice tests and true-false tests. However, the practical usefulness of these theoretical results remains doubtful. Equations that have been derived have not often been used in practical work in testing and test construction. One reason is that relatively little is known about how guessing combines with other sources of error variance that determine test reliability and what proportion of the total variance of test scores is accounted for by guessing. This article derives explicit formulas that allow for combinations of error variance due to guessing and other sources of error. These formulas provide a more realistic guide as to how much improvement in reliability can be expected by altering parameters such as number of test items, number of item choices, and the means and variances of examinees' observed scores.