A Comparison of Angoff, Yes/No and Ebel Standard Setting Methods

In this study, experts' decisions and cut-off scores obtained from Angoff, Yes/No and Ebel standard setting methods have been compared. The research data consist of 489 student scores obtained from a achievement test of 20 items and 17 experts' decisions, made according to three different standard setting methods for this achievement test. In the analysis of data, in order to set the compliance between experts, Kendall's coefficient of concordance (W) for Angoff and Ebel methods, and Cohen's Kappa statistics for Yes/No method have been calculated. Differences between the percentages of students, considered as successful according to three different standard setting methods, have been tested with the test of the difference between two correlated proportions. The students' scores have been tailored to binary as successful-unsuccessful according to three methods; and the compliance between them has been examined with Cohen's Kappa statistics. The compliance between the cutoff scores of three methods has been evaluated with Pearson Product-Moment Correlation Coefficient and the paired samples t test. At the end of this research, it is seen that the compliance of experts in the methods is at a reasonable level; there is significant difference on 0,01 error level among percentages of students considered as successful according to these three different standard setting methods and there is a compliance between students' scores that have been tailored to binary as successful-unsuccessful according to three methods. It is concluded that there is a high relationship between Angoff and Yes/No methods; there is moderate relationship between Angoff and Ebel methods; there is no significant relationship between Yes/No and Ebel methods.