The number of feedbacks needed for reliable evaluation. A multilevel analysis of the reliability, stability and generalisability of students' evaluation of teaching

被引:21
作者
Rantanen, Pekka [1 ,2 ]
机构
[1] HAAGA HELIA Univ Appl Sci, Helsinki, Finland
[2] Finnish Natl Board Educ, Helsinki, Finland
关键词
student evaluation of teaching; reliability; stability; generalisability; RATINGS; VALIDITY;
D O I
10.1080/02602938.2011.625471
中图分类号
G40 [教育学];
学科分类号
040101 ; 120403 ;
摘要
A multilevel analysis approach was used to analyse students' evaluation of teaching (SET). The low value of inter-rater reliability stresses that any solid conclusions on teaching cannot be made on the basis of single feedbacks. To assess a teacher's general teaching effectiveness, one needs to evaluate four randomly chosen course implementations. Two implementations are needed when one course is evaluated, and if one implementation is evaluated, up to 15 feedbacks are needed. The stability of students' ratings is very high, which reflects students' stable rating criteria. There is an obvious rating paradox: from the student's point of view, each rating is very precise, stable and justifiable, but from the teacher's point of view a single feedback reflects the quality of teaching to just a moderate extent. Cross-hierarchical analysis reveals that there are large discrepancies between the uses of rating scales; some students are systematically more lenient in their rating whereas others are systematically more severe. The study also reveals that some courses are generally rated more favourably and that some courses are more suitable for certain teachers. Managers can thus improve the quality of teaching by finding the most suitable courses for each teacher.
引用
收藏
页码:224 / 239
页数:16
相关论文
empty
未找到相关数据