Crowd-sourced assessment of technical skills: an opportunity for improvement in the assessment of laparoscopic surgical skills

被引：48
作者：
Deal, Shanley B. [1 ]
Lendvay, Thomas S. [2 ]
Haque, Mohamad I. [3 ]
Brand, Timothy [3 ]
Comstock, Bryan [2 ]
Warren, Justin [2 ]
Alseidi, Adnan [1 ]
机构：
[1] Virginia Mason Med Ctr, Dept Gen Surg, Seattle, WA 98101 USA
[2] Univ Washington, Dept Urol, Seattle, WA 98195 USA
[3] Madigan Army Med Ctr, Dept Gen Surg, Tacoma, WA 98431 USA
来源：
AMERICAN JOURNAL OF SURGERY | 2016年 / 211卷 / 02期
关键词：
Surgical skills education; Psychomotor skills; Surgical skills assessment; Crowd sourced data; ASSESSMENT-TOOL; SIMULATION; PERFORMANCE; SURGERY; FUNDAMENTALS; RESIDENTS;
D O I：
10.1016/j.amjsurg.2015.09.005
中图分类号：
R61 [外科手术学];
学科分类号：
摘要：
BACKGROUND: Objective, unbiased assessment of surgical skills remains a challenge in surgical education. We sought to evaluate the feasibility and reliability of Crowd-Sourced Assessment of Technical Skills. METHODS: Seven volunteer general surgery interns were given time for training and then testing, on laparoscopic peg transfer, precision cutting, and intracorporeal knot-tying. Six faculty experts (FEs) and 203 Amazon.com Mechanical Turk crowd workers (CWs) evaluated 21 deidentified video clips using the Global Objective Assessment of Laparoscopic Skills validated rating instrument. RESULTS: Within 19 hours and 15 minutes we received 662 eligible ratings from 203 CWs and 126 ratings from 6 FEs over 10 days. FE video ratings were of borderline internal consistency (Krippen-dorff's alpha = .55). FE ratings were highly correlated with CW ratings (Pearson's correlation coefficient = .78, P < .001). CONCLUSION: We propose the use of Crowd-Sourced Assessment of Technical Skills as a reliable, basic tool to standardize the evaluation of technical skills in general surgery. (C) 2016 Elsevier Inc. All rights reserved.
引用
收藏
页码：398 / 404
页数：7
相关论文
共 17 条
[1]   Laparoscopic skills training and assessment [J].
Aggarwal, R ;
Moorthy, K ;
Darzi, A .
BRITISH JOURNAL OF SURGERY, 2004, 91 (12) :1549-1558
[2]   Operative Experience of Residents in US General Surgery Programs A Gap Between Expectation and Experience [J].
Bell, Richard H., Jr. ;
Biester, Thomas W. ;
Tabuenca, Arnold ;
Rhodes, Robert S. ;
Cofer, Joseph B. ;
Britt, L. D. ;
Lewis, Frank R., Jr. .
ANNALS OF SURGERY, 2009, 249 (05) :719-724
[3]   Crowd-Sourced Assessment of Technical Skills: a novel method to evaluate surgical performance [J].
Chen, Carolyn ;
White, Lee ;
Kowalewski, Timothy ;
Aggarwal, Rajesh ;
Lintott, Chris ;
Comstock, Bryan ;
Kuksenok, Katie ;
Aragon, Cecilia ;
Holst, Daniel ;
Lendvay, Thomas .
JOURNAL OF SURGICAL RESEARCH, 2014, 187 (01) :65-71
[4]   My current thoughts on coefficient alpha and successor procedures [J].
Cronbach, LJ .
EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 2004, 64 (03) :391-418
[5]   Crowd-Sourced Assessment of Technical Skills: An Adjunct to Urology Resident Surgical Simulation Training [J].
Holst, Daniel ;
Kowalewski, Timothy M. ;
White, Lee W. ;
Brand, Timothy C. ;
Harper, Jonathan D. ;
Sorenson, Mathew D. ;
Kirsch, Sarah ;
Lendvay, Thomas S. .
JOURNAL OF ENDOUROLOGY, 2015, 29 (05) :604-609
[6]   Fundamentals of Laparoscopic Surgery: A Surgical Skills Assessment Tool in Gynecology [J].
Hur, Hye-Chun ;
Arden, Deborah ;
Dodge, Laura E. ;
Zheng, Bin ;
Ricciotti, Hope A. .
JSLS-JOURNAL OF THE SOCIETY OF LAPAROENDOSCOPIC SURGEONS, 2011, 15 (01) :21-26
[7]  
Krippendorff K., 2013, Content analysis, V3rd
[8]   RANDOM-EFFECTS MODELS FOR LONGITUDINAL DATA [J].
LAIRD, NM ;
WARE, JH .
BIOMETRICS, 1982, 38 (04) :963-974
[9]   Quantifying surgical skill: using the wisdom of crowds [J].
Lendvay, Thomas S. ;
White, Lee W. ;
Holst, Daniel ;
Kowalewski, Timothy ;
Harper, Jonathan D. ;
Sorensen, Mathew ;
Brand, Timothy C. ;
Truong, Mireille ;
Simpson, Khara ;
Smith, Roger .
JOURNAL OF THE AMERICAN COLLEGE OF SURGEONS, 2014, 219 (04) :E158-E159
[10]  
Martin JA, 1997, BRIT J SURG, V84, P273, DOI 10.1002/bjs.1800840237
12