Automated Technical Skill and Performance Assessment in Otology and Neurotology: A Scoping Review

被引：0
作者：
Nwosu, Obinna I. [1 ]
Ota, Mitsuki [3 ]
Goss, Deborah [1 ]
Crowson, Matthew G. [1 ,2 ]
机构：
[1] Massachusetts Eye & Ear, Dept Otolaryngol Head & Neck Surg, 243 Charles St, Boston, MA 02114 USA
[2] Harvard Med Sch, Dept Otolaryngol Head & Neck Surg, Boston, MA USA
[3] Harvard Med Sch, Boston, MA USA
来源：
OTOLOGY & NEUROTOLOGY | 2025年 / 46卷 / 03期
关键词：
Assessment; Automated; Neurotology; Otology; Surgical education; COMPETENCE ASSESSMENT-TOOL; OBJECTIVE ASSESSMENT; OTOLARYNGOLOGY; SIMULATION; FEEDBACK; SCALE; VALIDATION; BARRIERS; VIDEO;
D O I：
10.1097/MAO.0000000000004427
中图分类号：
R74 [神经病学与精神病学];
学科分类号：
摘要：
Objectives/HypothesisThis scoping review aims to provide an overview of existing semi-automated and fully automated methods for technical skill and performance assessment in otologic and neurotologic procedures.Study DesignScoping review.Databases ReviewedOvid MEDLINE (PubMed), Ovid EMBASE, Web of Science Core Collection, and IEEE Xplor Digital Library.MethodsA literature search was conducted according to PRISMA-ScR. Included studies were full-text articles that detailed an automated method of technical skill and performance assessment in otologic/neurotologic procedures. Extracted elements included general study characteristics (publication year, study objective, validity type, surgical procedure, and setting) and assessment approach characteristics (method of analysis, metrics assessed, source of metric data, degree of automation, and use of artificial intelligence [AI]).ResultsA total of 1,141 studies were identified from the literature search. After deduplication, title/abstract screening, and full-text review, 21 studies met the inclusion criteria. All but one of the included studies focused on mastoidectomy. Most studies assessed performance exclusively in VR-simulated mastoidectomy (n = 12) as opposed to cadaveric, 3D-printed, or live dissections. The majority of studies concentrated on establishing internal validity of their assessment methods (n = 13). Performance metrics were primarily obtained through motion analysis and final product analysis. Only a minority of studies used AI, which typically involved machine learning regression or classification to predict skill levels based on automatically extracted metrics.ConclusionThis scoping review explores the developing landscape of automated technical skill and performance assessment in otology and neurotology. Though progress has been made in automating assessment in the field, most investigations are narrowly focused on performance in VR-simulated mastoidectomy and lack external validity evidence. AI and computer vision (CV), which have advanced automated assessment in other surgical fields, have been underutilized in assessing performance in otology and neurotology. Future work must explore the development and validation of automated assessment approaches across a wider range of otologic and neurotologic procedures. Incorporation of novel AI/CV techniques may facilitate real-time integration of automated assessment in a broader range of simulated procedures and live surgical settings.
引用
收藏
页码：248 / 255
页数：8
相关论文
共 59 条
  • [1] Long D.M., Competency-based residency training: the next advance in graduate medical education, Acad Med J Assoc Am Med Coll, 75, pp. 1178-1183, (2000)
  • [2] Swing S.R., The ACGME outcome project: retrospective and prospective, Med Teach, 29, pp. 648-654, (2007)
  • [3] Ishman S.L., Brown D.J., Boss E.F., Et al., Development and pilot testing of an operative competency assessment tool for pediatric direct laryngoscopy and rigid bronchoscopy, Laryngoscope, 120, pp. 2294-2300, (2010)
  • [4] Stack B.C., Siegel E., Bodenner D., Carr M.M., A study of resident proficiency with thyroid surgery: creation of a thyroid-specific tool, Otolaryngol Head Neck Surg, 142, pp. 856-862, (2010)
  • [5] Lin S.Y., Laeeq K., Ishii M., Et al., Development and pilot-testing of a feasible, reliable, and valid operative competency assessment tool for endoscopic sinus surgery, Am J Rhinol Allergy, 23, pp. 354-359, (2009)
  • [6] Schwartz J., Costescu A., Mascarella M.A., Et al., Objective assessment of myringotomy and tympanostomy tube insertion: a prospective single-blinded validation study, Laryngoscope, 126, pp. 2140-2146, (2016)
  • [7] Piromchai P., Kasemsiri P., Wijewickrema S., Et al., The construct validity and reliability of an assessment tool for competency in cochlear implant surgery, Biomed Res Int, 2014, (2014)
  • [8] Obeid A.A., Al-Qahtani K.H., Ashraf M., Et al., Development and testing for an operative competency assessment tool for nasal septoplasty surgery, Am J Rhinol Allergy, 28, pp. e163-e167, (2014)
  • [9] Al-Qahtani K.H., Alkhalidi A.M., Islam T., Tool for assessing surgical tracheostomy skills in otolaryngology residents, B-ENT, 11, pp. 275-280, (2015)
  • [10] Laeeq K., Weatherly R.A., Masood H., Et al., Barriers to the implementation of competency-based education and assessment: a survey of otolaryngology program directors, Laryngoscope, 120, pp. 1152-1158, (2010)
123456