Consistency in teachers' judgments

Professional teachers at first stage of secondary school were asked to assess the relative weights of five dimensions that in Italy must be considered when the final grades of students are decided. The teachers were also asked to give an overall evaluation of two samples of typical (i.e., students who record similar performances on all the dimensions) vs. atypical students (i.e., students who exhibit poor performance on at least one dimension and good or excellent performances on the others). The results show that the teachers shared a weighting scheme for assessment of the importance of the five dimensions. However, their overall evaluations were consistent with this weighting scheme only with reference to typical students. When the teachers evaluated atypical students, their judgements were highly inconsistent. The paper briefly discusses the implications of these findings.