An Automated Skill Assessment Framework Based on Visual Motion Signals and a Deep Neural Network in Robot-Assisted Minimally Invasive Surgery

被引:4
作者
Pan, Mingzhang [1 ,2 ]
Wang, Shuo [1 ]
Li, Jingao [1 ]
Li, Jing [1 ]
Yang, Xiuze [1 ]
Liang, Ke [1 ,3 ]
机构
[1] Guangxi Univ, Coll Mech Engn, Nanning 530004, Peoples R China
[2] State Key Lab Conservat & Utilizat Subtrop Agrobi, Nanning 530004, Peoples R China
[3] Guangxi Univ, Sch Mech Engn, Guangxi Key Lab Mfg Syst & Adv Mfg Technol, Nanning 530004, Peoples R China
关键词
robot-assisted minimally invasive surgery; surgical skill assessment; visual motion tracking; kernel correlation filter; residual neural network; TECHNICAL SKILL; SURGICAL SKILL; TRACKING;
D O I
10.3390/s23094496
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Surgical skill assessment can quantify the quality of the surgical operation via the motion state of the surgical instrument tip (SIT), which is considered one of the effective primary means by which to improve the accuracy of surgical operation. Traditional methods have displayed promising results in skill assessment. However, this success is predicated on the SIT sensors, making these approaches impractical when employing the minimally invasive surgical robot with such a tiny end size. To address the assessment issue regarding the operation quality of robot-assisted minimally invasive surgery (RAMIS), this paper proposes a new automatic framework for assessing surgical skills based on visual motion tracking and deep learning. The new method innovatively combines vision and kinematics. The kernel correlation filter (KCF) is introduced in order to obtain the key motion signals of the SIT and classify them by using the residual neural network (ResNet), realizing automated skill assessment in RAMIS. To verify its effectiveness and accuracy, the proposed method is applied to the public minimally invasive surgical robot dataset, the JIGSAWS. The results show that the method based on visual motion tracking technology and a deep neural network model can effectively and accurately assess the skill of robot-assisted surgery in near real-time. In a fairly short computational processing time of 3 to 5 s, the average accuracy of the assessment method is 92.04% and 84.80% in distinguishing two and three skill levels. This study makes an important contribution to the safe and high-quality development of RAMIS.
引用
收藏
页数:17
相关论文
共 44 条
  • [1] Training and simulation for patient safety
    Aggarwal, Rajesh
    Mytton, Oliver T.
    Derbrew, Milliard
    Hananel, David
    Heydenburg, Mark
    Issenberg, Barry
    MacAulay, Catherine
    Mancini, Mary Elizabeth
    Morimoto, Takeshi
    Soper, Nathaniel
    Ziv, Amitai
    Reznick, Richard
    [J]. QUALITY & SAFETY IN HEALTH CARE, 2010, 19 : i34 - 43
  • [2] A Dataset and Benchmarks for Segmentation and Recognition of Gestures in Robotic Surgery
    Ahmidi, Narges
    Tao, Lingling
    Sefati, Shahin
    Gao, Yixin
    Lea, Colin
    Haro, Benjamin Bejar
    Zappella, Luca
    Khudanpur, Sanjeev
    Vidal, Rene
    Hager, Gregory D.
    [J]. IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, 2017, 64 (09) : 2025 - 2041
  • [3] [Anonymous], VINC SURG SYST
  • [4] Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating
    Azari, David P.
    Frasier, Lane L.
    Quamme, Sudha R. Pavuluri
    Greenberg, Caprice C.
    Pugh, Carla M.
    Greenberg, Jacob A.
    Radwin, Robert G.
    [J]. ANNALS OF SURGERY, 2019, 269 (03) : 574 - 581
  • [5] Analysis of the video motion tracking system "Kinovea" to assess surgical movements during robot-assisted radical prostatectomy
    Beulens, Alexander J. W.
    Namba, Hanae F.
    Brinkman, Willem M.
    Meijer, Richard P.
    Koldewijn, Evert L.
    Hendrikx, Ad J. M.
    van Basten, Jean-Paul
    van Merrienboer, Jeroen J. G.
    van der Poel, Henk G.
    Bangma, Chris
    Wagner, Cordula
    [J]. INTERNATIONAL JOURNAL OF MEDICAL ROBOTICS AND COMPUTER ASSISTED SURGERY, 2020, 16 (02)
  • [6] Surgical Skill and Complication Rates after Bariatric Surgery
    Birkmeyer, John D.
    Finks, Jonathan F.
    O'Reilly, Amanda
    Oerline, Mary
    Carlin, Arthur M.
    Nunn, Andre R.
    Dimick, Justin
    Banerjee, Mousumi
    Birkmeyer, Nancy J. O.
    [J]. NEW ENGLAND JOURNAL OF MEDICINE, 2013, 369 (15) : 1434 - 1442
  • [7] Darzi A, 2001, QUAL HEALTH CARE, V10, P64
  • [8] The minimally acceptable classification criterion for surgical skill: intent vectors and separability of raw motion data
    Dockter, Rodney L.
    Lendvay, Thomas S.
    Sweet, Robert M.
    Kowalewski, Timothy M.
    [J]. INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY, 2017, 12 (07) : 1151 - 1159
  • [9] Ebina K, 2021, SURG ENDOSC, V35, P4399, DOI 10.1007/s00464-020-07940-7
  • [10] Analysis of motion in laparoscopy: the deconstruction of an intracorporeal suturing task
    Farcas, Monica A.
    Trudeau, Maeve O'Neill
    Nasr, Ahmed
    Gerstle, J. Ted
    Carrillo, Brian
    Azzie, Georges
    [J]. SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES, 2017, 31 (08): : 3130 - 3139