Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator

Purpose: A validated high fidelity simulation model would provide a safe environment to teach thoracoscopic EA/TEF repair to novices. The study purpose was to evaluate validity evidence for performance measures on an EA/TEF simulator. Methods: IRB-exempt data were collected from 12 self-reported "novice" and 8 "experienced" pediatric surgeons. Participants evaluated the EA/TEF repair simulator using survey ratings that were analyzed for test content validity evidence. Additionally, deidentified operative performances were videotaped and independently rated by two surgeons using the Objective Structured Assessment for Technical Skills (OSATS) instrument. Novice and experienced OSATS were compared with p < .05 significant. Results: Participants had high overall simulator ratings. Internal structure was supported by high interitem consistency (a = .95 and .96) and interrater agreement (ICC) [.52, .84] for OSATS ratings. Experienced surgeons performed at a significantly higher level than novices for all five primary and two supplemental OSATS items (p < .05). Conclusion: Favorable participant ratings indicate the simulator is relevant to clinical practice and valuable as a learning tool. Further, performance ratings can discriminate experienced and novice performances of EA/TEF repair. These findings support the use of the simulator for performance assessment, representing the first validated measures from a simulator intended for pediatric surgical training. (C) 2014 Elsevier Inc. All rights reserved.