Does One Size Fit All? Examining the Application of Neurosurgery Residency Milestones Developed in the United States to a Taiwanese Culture

BACKGROUND: The Milestone Project was launched in 2009, charging specialties to develop specific educational accomplishments required to establish clinical competency. The milestone assessment method was first introduced to Taiwan in 2013 and before applying milestone assessments to our medical education system, the validity and reliability of these questionnaires needed to be evaluated. METHODS: Twenty neurosurgical faculty members representing 3 clinical divisions and all 4 branch institutes completed milestone questionnaires for 26 residents semiannually, resulting in 435 resident assessments being collected and analyzed. RESULTS: Cronbach's a, Kuder-Richardson Formula 20, and Kendall's W were used to show acceptable reliability and validity. Rater consistencies for nonskilled parts found that rater consistency progressively improved with time. Not all raters were able to assess the residents for the skilled parts resulting in nonassessable rates ranging from 9.5%-89.4%. For nonskilled and skilled items, milestone level as assessed by the staff improved as the resident progressed from residency year 3 (R3) to R6 in the residency program and showed that the milestone achievement level for an R3 was lower than that of an R6. CONCLUSIONS: Milestone assessments have high reliability and may be a helpful assessment tool. Although milestone assessment can provide thorough feedback concerning performance and the content of the training program, they may not perfectly suit all residency-training programs, especially in different countries or different cultures. Modifications should be done before applying milestones to different areas; therefore the results can truly reflect the progress and condition of the training and learning process.