How IRT Can Solve Problems of Ipsative Data in Forced-Choice Questionnaires

被引:122
作者
Brown, Anna [1 ]
Maydeu-Olivares, Alberto [2 ]
机构
[1] Univ Kent, Sch Psychol, Canterbury CT2 7NP, Kent, England
[2] Univ Barcelona, Fac Psychol, Barcelona, Spain
关键词
forced-choice format; ipsative data; multidimensional IRT; Thurstonian IRT model; Thurstonian factor model; MULTIDIMENSIONAL PAIRWISE PREFERENCE; RESPONSE STYLES; PERSONALITY; SCALES; BIAS;
D O I
10.1037/a0030641
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
In. multidimensional forced-choice (MFC) questionnaires, items measuring different attributes are presented in blocks, and participants have to rank order the items within each block (fully or partially). Such comparative formats can reduce the impact of numerous response biases often affecting single-stimulus items (aka rating or Likert scales). However, if scored with traditional methodology, MFC instruments produce ipsative data, whereby all individuals have a common total test score. Ipsative scoring distorts individual profiles (it is impossible to achieve all high or all low scale scores), construct validity (covariances between scales must sum to zero), criterion-related validity (validity coefficients must sum to zero), and reliability estimates. We argue that these problems are caused by inadequate scoring of forced-choice items and advocate the use of item response theory (IRT) models based on an appropriate response process for comparative data, such as Thurstone's law of comparative judgment. We show that when Thurstonian IRT modeling is applied (Brown & Maydeu-Olivares, 2011), even existing forced-choice questionnaires with challenging features can be scored adequately and that the IRT-estimated scores are free from the problems of ipsative data.
引用
收藏
页码:36 / 52
页数:17
相关论文
共 51 条
[1]  
Ackerman TA, 2005, MULTIVAR APPL SER, P3
[2]  
Asparouhov T., 2010, MULTIPLE IMPUTATION
[3]   Strengths and limitations of ipsative measurement [J].
Baron, H .
JOURNAL OF OCCUPATIONAL AND ORGANIZATIONAL PSYCHOLOGY, 1996, 69 :49-56
[4]   Increasing validity with forced-choice criterion measurement formats [J].
Bartram, Dave .
INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT, 2007, 15 (03) :263-272
[5]  
Brown A., 2009, C SOC IND ORG PSYCH
[6]   Satisfaction in Child and Adolescent Mental Health Services: Translating Users' Feedback into Measurement [J].
Brown, Anna ;
Ford, Tamsin ;
Deighton, Jessica ;
Wolpert, Miranda .
ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH, 2014, 41 (04) :434-446
[7]   Fitting a Thurstonian IRT model to forced-choice data using Mplus [J].
Brown, Anna ;
Maydeu-Olivares, Alberto .
BEHAVIOR RESEARCH METHODS, 2012, 44 (04) :1135-1147
[8]   Item Response Modeling of Forced-Choice Questionnaires [J].
Brown, Anna ;
Maydeu-Olivares, Alberto .
EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 2011, 71 (03) :460-502
[9]   Covariance structure analysis of ordinal ipsative data [J].
Chan, W ;
Bentler, PM .
PSYCHOMETRIKA, 1998, 63 (04) :369-399
[10]  
Chan W., 2003, Behaviormetrika, V30, P99, DOI [10.2333/bhmk.30.99, DOI 10.2333/BHMK.30.99]