Visual detection and tracking algorithms for minimally invasive surgical instruments: A comprehensive review of the state-of-the-art

被引:43
作者
Wang, Yan [1 ,2 ]
Sun, Qiyuan [1 ,2 ]
Liu, Zhenzhong [1 ,2 ]
Gu, Lin [3 ]
机构
[1] Tianjin Univ Technol, Sch Mech Engn, Tianjin Key Lab Adv Mechatron Syst Design & Intel, Tianjin 300384, Peoples R China
[2] Tianjin Univ Technol, Natl Demonstrat Ctr Expt Mech & Elect Engn Educ, Sch Mech Engn, Tianjin 300384, Peoples R China
[3] Univ Tokyo, RIKEN AIP, Tokyo 1030027, Japan
基金
中国国家自然科学基金;
关键词
Surgical robots; Machine vision; Detection and tracking of surgical; instruments; Deep learning; Feature extraction; CONVOLUTIONAL NEURAL-NETWORKS; OBJECT TRACKING; IMAGE-ANALYSIS; SURGERY; TOOLS;
D O I
10.1016/j.robot.2021.103945
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Minimally invasive surgical instrument visual detection and tracking is one of the core algorithms of minimally invasive surgical robots. With the development of machine vision and robotics, related technologies such as virtual reality, three-dimensional reconstruction, path planning, and human- machine collaboration can be applied to surgical operations to assist clinicians or use surgical robots to complete clinical operations. The minimally invasive surgical instrument vision detection and tracking algorithm analyzes the image transmitted by the surgical robot endoscope, extracting the position of the surgical instrument tip in the image, so as to provide the surgical navigation. This technology can greatly improve the accuracy and success rate of surgical operations. The purpose of this paper is to further study the visual detection and tracking technology of minimally invasive surgical instruments, summarize the existing research results, and apply it to the surgical robot project. By reading the literature, the author summarized the theoretical basis and related algorithms of this technology in recent years. Finally, the author compares the accuracy, speed and application scenario of each algorithm, and analyzes the advantages and disadvantages of each algorithm. The papers included in the review were selected through Web of Science, Google Scholar, PubMed and CNKI searches using the keywords: "object detection ", "object tracking ", "surgical tool detection ", "surgical tool tracking ", "surgical instrument detection "and "surgical instrument tracking "limiting results to the year range 1985-2021. Our study shows that this technology will have a great development prospect in the aspects of accuracy and real-time improvement in the future. (C)& nbsp;2021 Elsevier B.V. All rights reserved.
引用
收藏
页数:23
相关论文
共 113 条
  • [21] Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery
    Du, Xiaofei
    Allan, Maximilian
    Dore, Alessio
    Ourselin, Sebastien
    Hawkes, David
    Kelly, John D.
    Stoyanov, Danail
    [J]. INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY, 2016, 11 (06) : 1109 - 1119
  • [22] Felzenszwalb P, 2008, PROC CVPR IEEE, P1984
  • [23] García-Peraza-Herrera LC, 2017, IEEE INT C INT ROBOT, P5717, DOI 10.1109/IROS.2017.8206462
  • [24] Real-Time Segmentation of Non-rigid Surgical Tools Based on Deep Learning and Tracking
    Garcia-Peraza-Herrera, Luis C.
    Li, Wenqi
    Gruijthuijsen, Caspar
    Devreker, Alain
    Attilakos, George
    Deprest, Jan
    Vander Poorten, Emmanuel
    Stoyanov, Danail
    Vercauteren, Tom
    Ourselin, Sebastien
    [J]. COMPUTER-ASSISTED AND ROBOTIC ENDOSCOPY, 2017, 10170 : 84 - 95
  • [25] Girshick R., 2015, PROC ADVNEURAL INF P, P1440
  • [26] Rich feature hierarchies for accurate object detection and semantic segmentation
    Girshick, Ross
    Donahue, Jeff
    Darrell, Trevor
    Malik, Jitendra
    [J]. 2014 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2014, : 580 - 587
  • [27] Guan Hao, 2017, Journal of Chinese Computer Systems, V38, P147
  • [28] Haase S., 2013, 2013 IEEE WORKSH APP, DOI [10.1109/WACV.2013.6475053, DOI 10.1109/WACV.2013.6475053]
  • [29] Struck: Structured Output Tracking with Kernels
    Hare, Sam
    Golodetz, Stuart
    Saffari, Amir
    Vineet, Vibhav
    Cheng, Ming-Ming
    Hicks, Stephen L.
    Torr, Philip H. S.
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2016, 38 (10) : 2096 - 2109
  • [30] Detection, segmentation, and 3D pose estimation of surgical tools using convolutional neural networks and algebraic geometry
    Hasan, Md Kamrul
    Calvet, Lilian
    Rabbani, Navid
    Bartoli, Adrien
    [J]. MEDICAL IMAGE ANALYSIS, 2021, 70