Assessment of competence in video-assisted thoracoscopic surgery lobectomy: A Danish nationwide study

Background: Competence in video-assisted thoracoscopic surgery lobectomy has previously been established on the basis of numbers of procedures performed, but this approach does not ensure competence. Specific assessment tools, such as the newly developed video-assisted thoracoscopic surgery lobectomy assessment tool, allow for structured and objective assessment of competence. Our aim was to provide validity evidence for the video-assisted thoracoscopic surgery lobectomy assessment tool. Methods: Video recordings of 60 video-assisted thoracoscopic surgery lobectomies performed by 18 thoracic surgeons were rated using the video-assisted thoracoscopic surgery lobectomy assessment tool. All 4 centers of thoracic surgery in Denmark participated in the study. Two video-assisted thoracoscopic surgery experts rated the videos. They were blinded to surgeon and center. Results: The total internal consistency reliability Cronbach's alpha was 0.93. Inter-rater reliability between the 2 raters was Pearson's r = 0.71 (P < .001). The mean video-assisted thoracoscopic surgery lobectomy assessment tool scores for the 10 procedures performed by beginners were 22.1 (standard deviation [SD], 8.6) for the 28 procedures performed by the intermediate surgeons, 31.2 (SD, 4.4), and for the 20 procedures performed by experts 35.9 (SD, 2.9) (P < .001). Bonferroni post hoc tests showed that experts were significantly better than intermediates (P < .008) and beginners (P < .001). Intermediates' mean scores were significantly better than beginners (P < .001). The pass/fail standard calculated using the contrasting group's method was 31 points. One of the beginners passed, and 2 experts failed the test. Conclusions: Validity evidence was provided for a newly developed assessment tool for video-assisted thoracoscopic surgery lobectomy (video-assisted thoracoscopic surgery lobectomy assessment tool) in a clinical setting. The discriminatory ability among expert surgeons, intermediate surgeons, and beginners proved highly significant. The video-assisted thoracoscopic surgery lobectomy assessment tool could be an important aid in the future training and certification of thoracic surgeons.