A New Multidimensional Computerized Testing Approach: On-the-Fly Assembled Multistage Adaptive Testing Based on Multidimensional Item Response Theory

被引:0
作者
Li, Jingwen [1 ]
Sun, Jianan [1 ]
Shao, Mingyu [1 ]
Lai, Yinghui [2 ]
Chen, Chen [1 ]
机构
[1] Beijing Forestry Univ, Coll Sci, Beijing 100083, Peoples R China
[2] Hunan Inst Sci & Technol, Sch Educ Sci, Yueyang 414006, Peoples R China
基金
中国国家自然科学基金;
关键词
multidimensional item response theory; on-the-fly assembled multistage adaptive testing; ability point estimation; ability confidence ellipsoid; ability estimation accuracy; item exposure control; SELECTION; CAT;
D O I
10.3390/math13040594
中图分类号
O1 [数学];
学科分类号
0701 ; 070101 ;
摘要
Unidimensional on-the-fly assembled multistage adaptive testing (OMST), a flexible testing method, integrates the strengths of the adaptive test assembly of computerized adaptive testing (CAT) and the modular test administration of multistage adaptive testing (MST). Since numerous latent trait structures in practical applications are inherently multidimensional, extending the realm from unidimensional to multidimensional is necessary. Multidimensional item response theory (MIRT), a branch of mathematical and statistical latent variable modeling research, has an important position in the international testing field. Based on MIRT, this study proposes an approach of multidimensional OMST (OMST-M), and on-the-fly automated test assembly algorithms are proposed based on point estimation and confidence ellipsoid, respectively. OMST-M can effectively and flexibly measure multidimensional latent traits through stage-by-stage adaptive testing. The simulation results indicated that under different settings of latent trait structures, module lengths, and module contents, the OMST-M approach demonstrated good performance in terms of ability estimation accuracy and item exposure control. The empirical research revealed that the OMST-M approach was comparable to both multidimensional MST and CAT in ability estimation accuracy and exhibited remarkable flexibility in adjusting the length and content across its test stages. In summary, the proposed OMST-M features relatively high measurement accuracy, efficiency, convenient implementation, and practical feasibility.
引用
收藏
页数:27
相关论文
共 47 条
[1]   Validation of Computerized Adaptive Testing in an Outpatient Nonacademic Setting: The VOCATIONS Trial [J].
Achtyes, Eric D. ;
Halstead, Scott ;
Smart, Leann ;
Moore, Tara ;
Frank, Ellen ;
Kupfer, David J. ;
Gibbons, Robert .
PSYCHIATRIC SERVICES, 2015, 66 (10) :1091-1096
[2]   Digital transformation in German higher education: student and teacher perceptions and usage of digital media [J].
Bond, Melissa ;
Marin, Victoria I. ;
Dolch, Carina ;
Bedenlier, Svenja ;
Zawacki-Richter, Olaf .
INTERNATIONAL JOURNAL OF EDUCATIONAL TECHNOLOGY IN HIGHER EDUCATION, 2018, 15
[3]   Item Response Theory [J].
Cai, Li ;
Choi, Kilchan ;
Hansen, Mark ;
Harrell, Lauren .
ANNUAL REVIEW OF STATISTICS AND ITS APPLICATION, VOL 3, 2016, 3 :297-321
[4]   Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications [J].
Chalmers, R. Philip .
JOURNAL OF STATISTICAL SOFTWARE, 2016, 71 (05)
[5]  
Chalmers RP, 2012, J STAT SOFTW, V48, P1
[6]   a-stratified multistage computerized adaptive testing [J].
Chang, HH ;
Ying, ZL .
APPLIED PSYCHOLOGICAL MEASUREMENT, 1999, 23 (03) :211-222
[7]   To weight or not to weight? Balancing influence of initial items in adaptive testing [J].
Chang, Hua-Hua ;
Ying, Zhiliang .
PSYCHOMETRIKA, 2008, 73 (03) :441-450
[8]   Statistical Applications in Educational Measurement [J].
Chang, Hua-Hua ;
Wang, Chun ;
Zhang, Susu .
ANNUAL REVIEW OF STATISTICS AND ITS APPLICATION, VOL 8, 2021, 2021, 8 :439-461
[9]   Psychometrics Behind Computerized Adaptive Testing [J].
Chang, Hua-Hua .
PSYCHOMETRIKA, 2015, 80 (01) :1-20
[10]   Nonparametric CAT for CD in Educational Settings With Small Samples [J].
Chang, Yuan-Pei ;
Chiu, Chia-Yi ;
Tsai, Rung-Ching .
APPLIED PSYCHOLOGICAL MEASUREMENT, 2019, 43 (07) :543-561