Increasing validity with forced-choice criterion measurement formats

The relative validities of forced-choice (ipsative) and Likert rating-scale item formats as criterion measures are examined. While there has been much debate about the relative technical and psychometric merits and demerits of ipsative instruments, the present research focused on the crucial question of whether the use of this format has any practical benefit - in terms of improved validity. An analysis is reported from a meta-analysis data set. This demonstrates that higher operational validity coefficients (prediction of line-manager ratings of competencies) are associated with the use of forced-choice (r =.38) rather than rating scale (r =.25) item formats for the criterion measurement instrument when performance is rated by the same line managers on both formats and where the predictor is held constant. Thus the apparent criterion-related validity of a predictor can increase by 50% simply by changing the format of the criterion measurement instrument. The implications of this for practice are discussed.