Catheter segmentation in X-ray fluoroscopy using synthetic data and transfer learning with light U -nets

被引:38
作者
Gherardini, Marta [1 ,2 ]
Mazomenos, Evangelos [3 ,4 ]
Menciassi, Arianna [1 ,2 ]
Stoyanov, Danail [3 ]
机构
[1] Scuola Super Sant Anna, BioRobot Inst, Pisa, Italy
[2] Scuola Super SantaAZAnna, Dept Excellence Robot & AI, I-56127 Pisa, Italy
[3] UCL, Wellcome EPSRC Ctr Intervent & Surg Sci, London, England
[4] Univ Leeds, Sch Elect & Elect Engn, Leeds, W Yorkshire, England
基金
英国工程与自然科学研究理事会; 欧盟地平线“2020”;
关键词
3D TRANSESOPHAGEAL ECHOCARDIOGRAPHY; CONVOLUTIONAL NEURAL-NETWORKS; GUIDEWIRE TRACKING;
D O I
10.1016/j.cmpb.2020.105420
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Background and objectivesAutomated segmentation and tracking of surgical instruments and catheters under X-ray fluoroscopy hold the potential for enhanced image guidance in catheter-based endovascular procedures. This article presents a novel method for real-time segmentation of catheters and guidewires in 2d X-ray images. We employ Convolutional Neural Networks (CNNs) and propose a transfer learning approach, using synthetic fluoroscopic images, to develop a lightweight version of the U-Net architecture. Our strategy, requiring a small amount of manually annotated data, streamlines the training process and results in a U-Net model, which achieves comparable performance to the state-of-the-art segmentation, with a decreased number of trainable parameters. MethodsThe proposed transfer learning approach exploits high-fidelity synthetic images generated from real fluroscopic backgrounds. We implement a two-stage process, initial end-to-end training and fine-tuning, to develop two versions of our model, using synthetic and phantom fluoroscopic images independently. A small number of manually annotated in-vivo images is employed to fine-tune the deepest 7 layers of the U-Net architecture, producing a network specialized for pixel-wise catheter/guidewire segmentation. The network takes as input a single grayscale image and outputs the segmentation result as a binary mask against the background. ResultsEvaluation is carried out with images from in-vivo fluoroscopic video sequences from six endovascular procedures, with different surgical setups. We validate the effectiveness of developing the U-Net models using synthetic data, in tests where fine-tuning and testing in-vivo takes place both by dividing data from all procedures into independent fine-tuning/testing subsets as well as by using different in-vivo sequences. Accurate catheter/guidewire segmentation (average Dice coefficient of ~ 0.55, ~ 0.26 and ~ 0.17) is obtained with both U-Net models. Compared to the state-of-the-art CNN models, the proposed U-Net achieves comparable performance ( ± 5% average Dice coefficients) in terms of segmentation accuracy, while yielding a 84% reduction of the testing time. This adds flexibility for real-time operation and makes our network adaptable to increased input resolution. ConclusionsThis work presents a new approach in the development of CNN models for pixel-wise segmentation of surgical catheters in X-ray fluoroscopy, exploiting synthetic images and transfer learning. Our methodology reduces the need for manually annotating large volumes of data for training. This represents an important advantage, given that manual pixel-wise annotations is a key bottleneck in developing CNN segmentation models. Combined with a simplified U-Net model, our work yields significant advantages compared to current state-of-the-art solutions. © 2020
引用
收藏
页数:11
相关论文
共 37 条
[1]  
Adebar Troy K., 2013, Information Processing in Computer-Assisted Interventions. Proceedings of 4th International Conference (IPCAI 2013): LNCS 7915, P61, DOI 10.1007/978-3-642-38568-1_7
[2]  
Ambrosini Pierre, 2017, Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017. 20th International Conference. Proceedings: LNCS 10434, P577, DOI 10.1007/978-3-319-66185-8_65
[3]   A Hidden Markov Model for 3D Catheter Tip Tracking With 2D X-ray Catheterization Sequence and 3D Rotational Angiography [J].
Ambrosini, Pierre ;
Smal, Ihor ;
Ruijters, Daniel ;
Niessen, Wiro J. ;
Moelker, Adriaan ;
Van Walsum, Theo .
IEEE TRANSACTIONS ON MEDICAL IMAGING, 2017, 36 (03) :757-768
[4]  
[Anonymous], ARXIV170310722
[5]   Guide-wire tracking during endovascular interventions [J].
Baert, SAM ;
Viergever, MA ;
Niessen, WJ .
IEEE TRANSACTIONS ON MEDICAL IMAGING, 2003, 22 (08) :965-972
[6]  
Balzer Jan, 2016, Interv Cardiol, V11, P59, DOI 10.15420/icr.2016.11.1.59
[7]  
Baur Christoph, 2016, Medical Imaging and Augmented Reality. 7th International Conference, MIAR 2016. Proceedings: LNCS 9805, P38, DOI 10.1007/978-3-319-43775-0_4
[8]   Clinical feasibility of x-ray based pose estimation of a transthoracic echo probe using attached fiducials [J].
Bodart, Lindsay E. ;
Ciske, Benjamin R. ;
Wagner, Martin ;
Raval, Amish N. ;
Speidel, Michael A. .
MEDICAL IMAGING 2018: IMAGE-GUIDED PROCEDURES, ROBOTIC INTERVENTIONS, AND MODELING, 2018, 10576
[9]   Robust Catheter and Guidewire Tracking Using B-Spline Tube Model and Pixel-Wise Posteriors [J].
Chang, Ping-Lin ;
Rolls, Alexander ;
De Praetere, Herbert ;
Poorten, Emmanuel Vander ;
Riga, Celia V. ;
Bicknell, Colin D. ;
Stoyanov, Danail .
IEEE ROBOTICS AND AUTOMATION LETTERS, 2016, 1 (01) :303-308
[10]   GUIDEWIRE TRACKING USING A NOVEL SEQUENTIAL SEGMENT OPTIMIZATION METHOD IN INTERVENTIONAL X-RAY VIDEOS [J].
Chen, Bor-Jeng ;
Wu, Ziyan ;
Sun, Shanhui ;
Zhang, Dong ;
Chen, Terrence .
2016 IEEE 13TH INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING (ISBI), 2016, :103-106