Deep learning-based monocular placental pose estimation: towards collaborative robotics in fetoscopy

被引:20
作者
Ahmad, Mirza Awais [1 ]
Ourak, Mouloud [1 ]
Gruijthuijsen, Caspar [1 ]
Deprest, Jan [2 ,3 ]
Vercauteren, Tom [4 ]
Vander Poorten, Emmanuel [1 ]
机构
[1] Katholieke Univ Leuven, Dept Mech Engn, Leuven, Belgium
[2] Katholieke Univ Leuven, Dept Dev & Regenerat, Leuven, Belgium
[3] UZ Leuven, Dept Obstet & Gynaecol, Leuven, Belgium
[4] Kings Coll London, Sch Biomed Engn & Imaging Sci, London, England
基金
英国工程与自然科学研究理事会; 英国惠康基金;
关键词
Deep learning; Convolutional neural networks; Robot assisted surgery; Shared control; Orientation estimation; Fetoscopy; TWIN TRANSFUSION SYNDROME; LASER-SURGERY; MANIPULATOR;
D O I
10.1007/s11548-020-02166-3
中图分类号
R318 [生物医学工程];
学科分类号
0831 ;
摘要
Purpose Twin-to-twin transfusion syndrome (TTTS) is a placental defect occurring in monochorionic twin pregnancies. It is associated with high risks of fetal loss and perinatal death. Fetoscopic elective laser ablation (ELA) of placental anastomoses has been established as the most effective therapy for TTTS. Current tools and techniques face limitations in case of more complex ELA cases. Visualization of the entire placental surface and vascular equator; maintaining an adequate distance and a close to perpendicular angle between laser fiber and placental surface are central for the effectiveness of laser ablation and procedural success. Robot-assisted technology could address these challenges, offer enhanced dexterity and ultimately improve the safety and effectiveness of the therapeutic procedures. Methods This work proposes a 'minimal' robotic TTTS approach whereby rather than deploying a massive and expensive robotic system, a compact instrument is 'robotised' and endowed with 'robotic' skills so that operators can quickly and efficiently use it. The work reports on automatic placental pose estimation in fetoscopic images. This estimator forms a key building block of a proposed shared-control approach for semi-autonomous fetoscopy. A convolutional neural network (CNN) is trained to predict the relative orientation of the placental surface from a single monocular fetoscope camera image. To overcome the absence of real-life ground-truth placenta pose data, similar to other works in literature (Handa et al. in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016; Gaidon et al. in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016; Vercauteren et al. in: Proceedings of the IEEE, 2019) the network is trained with data generated in a simulated environment and an in-silico phantom model. A limited set of coarsely manually labeled samples from real interventions are added to the training dataset to improve domain adaptation. Results The trained network shows promising results on unseen samples from synthetic, phantom and in vivo patient data. The performance of the network for collaborative control purposes was evaluated in a virtual reality simulator in which the virtual flexible distal tip was autonomously controlled by the neural network. Conclusion Improved alignment was established compared to manual operation for this setting, demonstrating the feasibility to incorporate a CNN-based estimator in a real-time shared control scheme for fetoscopic applications.
引用
收藏
页码:1561 / 1571
页数:11
相关论文
共 25 条
[1]  
Ahmad M. A., 2019, DESIGN SHARED CONTRO
[2]  
[Anonymous], 2016, PROC CVPR IEEE, DOI DOI 10.1109/CVPR.2016.470
[3]  
[Anonymous], 2004, The visualization toolkit: an object-oriented approach to 3D graphics
[4]  
[Anonymous], 2016, PROC CVPR IEEE, DOI DOI 10.1109/CVPR.2016.442
[5]  
DEPREST J, 2004, ENDOSCOPY FETAL MED
[6]   Alternative technique for Nd:YAG laser coagulation in twin-to-twin transfusion syndrome with anterior placenta [J].
Deprest, JA ;
Van Schoubroeck, D ;
Van Ballaer, PP ;
Flageole, H ;
Van Assche, FA ;
Vandenberghe, K .
ULTRASOUND IN OBSTETRICS & GYNECOLOGY, 1998, 11 (05) :347-352
[7]   Fetal membrane heating after spontaneous and iatrogenic membrane rupture: A review of current evidence [J].
Devlieger, R. ;
Millar, L. K. ;
Bryant-Greenwood, G. ;
Lewi, L. ;
Deprest, J. A. .
AMERICAN JOURNAL OF OBSTETRICS AND GYNECOLOGY, 2006, 195 (06) :1512-1520
[8]   Stretching of fetal membranes increases the concentration of interleukin-8 and collagenase activity [J].
ElMaradny, E ;
Kanayama, N ;
Halim, A ;
Maehara, K ;
Terao, T .
AMERICAN JOURNAL OF OBSTETRICS AND GYNECOLOGY, 1996, 174 (03) :843-849
[9]   Bending laser manipulator for intrauterine surgery and viscoelastic model of fetal rat tissue [J].
Harada, Kanako ;
Bo, Zhang ;
Enosawa, Shin ;
Chiba, Toshio ;
Fujie, Masakatsu G. .
PROCEEDINGS OF THE 2007 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, VOLS 1-10, 2007, :611-616
[10]   A mixed-reality surgical trainer with comprehensive sensing for fetal laser minimally invasive surgery [J].
Javaux, Allan ;
Bouget, David ;
Gruijthuijsen, Caspar ;
Stoyanov, Danail ;
Vercauteren, Tom ;
Ourselin, Sebastien ;
Deprest, Jan ;
Denis, Kathleen ;
Vander Poorten, Emmanuel .
INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY, 2018, 13 (12) :1949-1957