Video assessment of basic surgical trainees' operative skills

BACKGROUND: We previously described a feasible, reliable, valid, and acceptable clinical assessment form for basic surgical trainees (BSTs). We now recently assessed tissue-handling skills using real-time assessment (RTA) and video assessment (VA) and addressed feasibility, reliability, validity, and trainer-trainee agreement using the same assessment form. METHODS: Nine BSTs were videotaped as they performed open inguinal hernia repairs at 6 and 12 months of surgical training. Edited videotapes were independently rated by 7 consultants and 5 trainees using the technical part of the Edinburgh BST Assessment Form (EBSTAF-Tech), the Toronto Global Rating Scale of Surgical Performance (Toronto scale), and a visual analogue scale. RESULTS: RTA and VA both proved to be reliable tools (RTA alpha >= 85; VA alpha >= .76; interclass correlation coefficient [ICC] >= .69, rho >= .694 [P >= .004]), but RTA was not construct valid. VA scores distinguished consultant from trainee operators (Mann-Whitney P < .02), with trainers able to discriminate between trainee levels (Wilcoxon P = .01 to .023). Concurrent validity was demonstrated by the VA (trainers tau-b = .71 to .79 [P < .001] and trainees tau-b = .79 to .82 [P < .001]) with good trainer-trainee agreement (EBSTAF-Tech tau-b = .35 [P = .03), Toronto tau-b = .46 [P = .006), and visual analogue scale tau-b = .46 [P = .006]). CONCLUSIONS: VA of BST tissue-handling skills is feasible, reliable, valid, and highly sensitive. It may also improve trainee self-assessment skills by promoting reflective practice. (c) 2008 Elsevier Inc. All rights reserved.