The Position of Distractors in Multiple-Choice Test Items: The Strongest Precede the Weakest

被引:6
作者
Lions, Severin [1 ]
Monsalve, Carlos [1 ]
Dartnell, Pablo [1 ,2 ,3 ]
Godoy, Maria Ines [4 ]
Cordova, Nora [4 ]
Jimenez, Daniela [4 ]
Blanco, Maria Paz [1 ]
Ortega, Gabriel [1 ]
Lemarie, Julie [5 ]
机构
[1] Univ Chile, Ctr Adv Res Educ FB0003, Inst Educ, Santiago, Chile
[2] Univ Chile, Ctr Math Modeling AFB170001, Santiago, Chile
[3] Univ Chile, Dept Math Engn, Santiago, Chile
[4] Univ Chile, Dept Evaluac Med & Registro Educ, Santiago, Chile
[5] Univ Toulouse, UT2J CNRS, CLLE Cognit Langues Langage Ergon, Toulouse, France
关键词
assessment; educational tests; multiple-choice; response placement; distractors; ACHIEVEMENT; CONSEQUENCES; IMPACT; FLAWS;
D O I
10.3389/feduc.2021.731763
中图分类号
G40 [教育学];
学科分类号
040101 ; 120403 ;
摘要
Middle bias has been reported for responses to multiple-choice test items used in educational assessment. It has been claimed that this response bias probably occurs because test developers tend to place correct responses among middle options, tests thus presenting a middle-biased distribution of answer keys. However, this response bias could be driven by strong distractors being more frequently located among middle options. In this study, the frequency of responses to a Chilean national examination used to rank students wanting to access higher education was used to categorize distractors based on attractiveness level. The distribution of different distractor types (best distractor, non-functioning distractors horizontal ellipsis ) was analyzed across 110 tests of 80 five-option items administered to assess several disciplines in five consecutive years. Results showed that the strongest distractors were more frequently found among middle options, most commonly at option C. In contrast, the weakest distractors were more frequently found at the last option (E). This pattern did not substantially vary across disciplines or years. Supplementary analyses revealed that a similar position bias for distractors could be observed in tests administered in countries other than Chile. Thus, the location of different types of distractors might provide an alternative explanation for the middle bias reported in literature for tests' responses. Implications for test developers, test takers, and researchers in the field are discussed.
引用
收藏
页数:6
相关论文
共 32 条
[1]   The Impact of item flaws, testing at low cognitive level, and low distractor functioning on multiple-choice question quality [J].
Ali, Syed Haris ;
Ruit, Kenneth G. .
PERSPECTIVES ON MEDICAL EDUCATION, 2015, 4 (05) :244-251
[2]  
Ambu-Saidi A., 2000, THESIS U GLASGOW
[3]   Guess where: The position of correct answers in multiple-choice test items as a psychometric variable [J].
Attali, Y ;
Bar-Hillel, M .
JOURNAL OF EDUCATIONAL MEASUREMENT, 2003, 40 (02) :109-128
[4]   Seek whence: Answer sequences and their consequences in key-balanced multiple-choice tests [J].
Bar-Hillel, M ;
Attali, Y .
AMERICAN STATISTICIAN, 2002, 56 (04) :299-303
[5]   Scoring and keying multiple choice tests: A case study in irrationality [J].
Maya Bar-Hillel ;
David Budescu ;
Yigal Attali .
Mind & Society, 2005, 4 (1) :3-12
[6]   Position Effects in Choice From Simultaneous Displays: A Conundrum Solved [J].
Bar-Hillel, Maya .
PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2015, 10 (04) :419-433
[7]  
Carnegie JA, 2017, CAN J SCHOLARSH TEA, V8
[8]   GENERAL RESPONSE PATTERN TO 5-CHOICE ITEMS [J].
CLARK, EL .
JOURNAL OF EDUCATIONAL PSYCHOLOGY, 1956, 47 (02) :110-117
[9]   Examining the effects of testwiseness in conceptual physics evaluations [J].
DeVore, Seth ;
Stewart, John ;
Stewart, Gay .
PHYSICAL REVIEW PHYSICS EDUCATION RESEARCH, 2016, 12 (02)
[10]   The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education [J].
Downing, SM .
ADVANCES IN HEALTH SCIENCES EDUCATION, 2005, 10 (02) :133-143