Multiple-true-false questions reveal more thoroughly the complexity of student thinking than multiple-choice questions: a Bayesian item response model comparison

被引:35
作者
Brassil, Chad E. [1 ]
Couch, Brian A. [1 ]
机构
[1] Univ Nebraska, Sch Biol Sci, 204 Manter, Lincoln, NE 68588 USA
关键词
Assessment; Bayesian analysis; Item response theory; Mixed conceptions; Multiple-choice; Multiple-true-false; Question format; Undergraduate; KNOWLEDGE; VALIDATION; PARAMETERS; ABILITY;
D O I
10.1186/s40594-019-0169-0
中图分类号
G40 [教育学];
学科分类号
040101 ; 120403 ;
摘要
Background: Within undergraduate science courses, instructors often assess student thinking using closed-ended question formats, such as multiple-choice (MC) and multiple-true-false (MTF), where students provide answers with respect to predetermined response options. While MC and MTF questions both consist of a question stem followed by a series of options, MC questions require students to select just one answer, whereas MTF questions enable students to evaluate each option as either true or false. We employed an experimental design in which identical questions were posed to students in either format and used Bayesian item response modeling to understand how responses in each format compared to inferred student thinking regarding the different options. Results: Our data support a quantitative model in which students approach each question with varying degrees of comprehension, which we label as mastery, partial mastery, and informed reasoning, rather than uniform random guessing. MTF responses more closely estimate the proportion of students inferred to have complete mastery of all the answer options as well as more accurately identify students holding misconceptions. The depth of instructional information elicited by MTF questions is demonstrated by the ability of MTF results to predict the MC results, but not vice-versa. We further discuss how MTF responses can be processed and interpreted by instructors. Conclusions: This research supports the hypothesis that students approach MC and MTF questions with varying levels of understanding and demonstrates that the MTF format has a greater capacity to characterize student thinking regarding the various response options.
引用
收藏
页数:17
相关论文
共 57 条
[1]   Development and Validation of Instruments to Measure Learning of Expert-Like Thinking [J].
Adams, Wendy K. ;
Wieman, Carl E. .
INTERNATIONAL JOURNAL OF SCIENCE EDUCATION, 2011, 33 (09) :1289-1312
[2]   An empirical investigation of the effects of three methods of handling guessing and risk taking on the psychometric indices of a test [J].
Alnabhan, M .
SOCIAL BEHAVIOR AND PERSONALITY, 2002, 30 (07) :645-652
[3]  
Angelo T.A., 1998, CLASSROOM ASSESSMENT
[4]  
[Anonymous], 2004, ITEM RESPONSE THEORY, DOI DOI 10.1201/9781482276725
[5]  
[Anonymous], 1992, The Physics Teacher, DOI [10.1119/1.2343497, DOI 10.1119/1.2343497]
[6]  
[Anonymous], 2012, Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering
[7]  
[Anonymous], 2005, Educational Measurement: Issues and Practice, DOI [10.1111/J.1745-3992.2005.00006.X, DOI 10.1111/J.1745-3992.2005.00006.X]
[8]   Personality, expectations, and response strategies in multiple-choice question examinations in university students: A test of Gray's hypotheses [J].
Avila, C ;
Torrubia, R .
EUROPEAN JOURNAL OF PERSONALITY, 2004, 18 (01) :45-59
[9]   Developing the theory of formative assessment [J].
Black, Paul ;
Wiliam, Dylan .
EDUCATIONAL ASSESSMENT EVALUATION AND ACCOUNTABILITY, 2009, 21 (01) :5-31
[10]  
BOCK RD, 1972, PSYCHOMETRIKA, V37, P29