Multi-source evaluation of interpersonal and communication skills of family medicine residents

There is a lack of information on the use of multi-source evaluation to assess trainees' interpersonal and communication skills in Oriental settings. This study is conducted to assess the reliability and applicability of assessing the interpersonal and communication skills of family medicine residents by patients, peer residents, nurses, and teaching staffs and to compare the ratings with the objective structured clinical examination (OSCE). Our results revealed instruments used by staffs, peers, nurses, and self-evaluation have good internal consistency reliability (alpha > 0.90), except for the behavioral checklist (alpha = 0.57). Staffs', peers', and nurses' evaluations were highly correlated with one another (r = 0.722 for staff- and peer-rating, r = 0.734 for staff- and nurse-rating, r = 0.634 for peer- and nurse-rating). However, residents' self-rating and patients-rating were not correlated to ratings by any other raters. OSCE evaluation was correlated to peer-rating (r = 0.533) and staff-rating (r = 0.642), but not correlated to self- or patient-rating. The generalizability study revealed the major sources of variance came from the types of rater and the interaction of residents and types of rater. This study found self-rating and patient-rating were not consistent with other sources of rating on residents' interpersonal and communication skills. Whether variations among different types of rater in a multi-source evaluation should be regarded as measurement errors or complementary information is worth further study.