Toward new approaches for evaluating student field performance: Tapping the implicit criteria used by experienced field instructors

This study determined the reliability of ratings and consistency of descriptions generated by experienced field instructors using only their acquired practice wisdom as a framework to evaluate students. Ten field instructors independently divided 20 student vignettes into as many categories as necessary to reflect various levels of student performance, described their categories, and ranked the individual vignettes within each category. The independently generated categories and their descriptions were very similar across instructors, and the inter-rater reliability was very high both for the placement of vignettes into categories (0.77) and the rankings (0.83).