Assessing the effectiveness of combining evaluation methods for the early identification of students with inadequate knowledge during a clerkship

This study examined the predictive validity of in-clerkship evaluation methods to identify medical students who have insufficient knowledge. Study subjects were 124 third-year medical students at the Uniformed Services University. Insufficient knowledge was defined by: (1) a clerkship 'pre-test' score one standard deviation below the mean or lower; or (2) any teacher verbally rating a student's general knowledge as 'marginal' or less; or (3) a student did not pass Step One of the United States Medical Licensing Examination (USMLE). We determined sensitivity and specificity using a standard score of 300 on the end of clerkship National Board of Medical Examiners (NBME) subject examination in medicine as the outcome variable. Sixteen students scored less than or equal to 300 on the NBME examination. The sensitivity of the 'pre-test' or verbal comments alone was 44% (seven of 16 students). By combining methods, 11 students were identified, for a sensitivity of 69%. The specificity of all methods was >90%. Using USMLE Step One pass-fail performance did not improve sensitivity. Combining a 'pre-test' and instructors' formal evaluation session comments improves the earl-v identification of students with insufficient knowledge, allowing for formative feedback and remediation during the clerkship.