Reliability and benefits of medical student peers in rating complex clinical skills

Background: Peers have been shown to be reliable raters in an objective structured clinical exam (OSCE). While the literature supports the use of senior level students in rating basic clinical skills, little is known about the reliability of peers of the same level of training in assessing complex clinical skills. Aim: To investigate the reliability of student peers of the same level of training in rating complex clinical skills in a geriatric OSCE. Methods: Peer (n = 115) ratings were compared to faculty ratings using correlation and generalizability analysis. Paired Wilcoxon Signed-Rank test was used to establish peer learning benefits. Results: Reliability of the OSCE was moderately strong (G-coefficient = 0.70) with strong correlations between peer and faculty ratings for the overall OSCE (r = 0.78, p = 0.001) and for each case (r = 0.70-0.85, p = 0.001). Generalizability analysis indicated that raters contributed minimally to score variance. Peers reported gaining learning benefits from the rating process. Conclusion: Peer raters of the same level of training can provide accurate ratings of complex clinical tasks and can serve as an important resource in assessing student performance in an OSCE. The peer review process can also serve an important role in enhancing student learning.