Assessing residents' clinical performance: Cumulative results of a four-year study with the Objective Structured Clinical Examination

Background. The Objective Structured Clinical Examination (OSCE) is an objective method for assessing clinical skills and can be used to identify deficits in clinical skill. During the past 5 years, we have administered 4 OSCEs to all general surgery residents and interns. Methods. Two OSCEs (1993 and 1994) were used as broad-based examinations of the core areas of general surgery; subsequent OSCEs (1995 and 1997) were used as needs assessments. For each year, the reliability of the entire examination was calculated with Cronbach's alpha. A reliability-based minimal competence score (MCS) was defined as the mean performance (in percent) minus the standard error of measurement for each group in 1997 (interns, junior residents, and senior residents). Results. The reliability of each OSCE was acceptable, ranging from 0.63 to 0.91. The MCS during the 4-year period ranged from 45% to 65%. In 1997, 4 interns, 2 junior residents, and 2 senior residents scored below their group's MCS. MCS for the groups increased across training levels in developmental fashion (P <.05). Conclusions. Given the relatively stable findings observed, we conclude (1) the OSCE can be used to identify group and individual differences reliably in clinical skills, and (2) we continue to use this method to develop appropriate curricular remediation for deficits in both individuals and groups.