ASSESS THE ASSESSMENT: AN AUTOMATED TOOL FOR ANALYZING MULTIPLE CHOICE EXAMS

Multiple choice exams are widely used at educational institutions. Besides many advantages (and some disadvantages), they offer the possibility to be easily analyzed by various statistical methods. The Item Response Theory provides several methods and models for this purpose. The most famous is the Rasch Model and its various extensions. Unfortunately, few lecturers (outside the field of mathematics, statistics or psychology) are familiar with these models and/or have the statistical knowledge to apply them. We developed a tool to automatically analyze multiple choice exams and the items within the exams, using several statistical methods like the mentioned Rasch Model and its extensions for polytomous items response categories but also more commonly used methods like hierarchical clustering, multidimensional scaling, factor analysis or analysis of variance. Together with the respective key figures the report gives detailed explanations, so that each user with some rather basic statistical knowledge can correctly interpret their meaning, thus enabling him to assess and improve the quality of his multiple choice exams and test items.