Surgical planning of pelvic tumor using multi-view CNN with relation-context representation learning

被引:24
作者
Qu, Yang [1 ]
Li, Xiaomin [1 ]
Yan, Zhennan [2 ]
Zhao, Liang [3 ]
Zhang, Lichi [4 ]
Liu, Chang [3 ]
Xie, Shuaining [3 ]
Li, Kang [5 ]
Metaxas, Dimitris [6 ]
Wu, Wen [7 ]
Hao, Yongqiang [7 ]
Dai, Kerong [7 ,8 ]
Zhang, Shaoting [3 ,9 ]
Tao, Xiaofeng [1 ]
Ai, Songtao [1 ]
机构
[1] Shanghai Jiao Tong Univ, Sch Med, Shanghai Peoples Hosp 9, Dept Radiol, Shanghai 200011, Peoples R China
[2] SenseBrain Technol, Princeton, NJ 08540 USA
[3] SenseTime Res, Shanghai 200233, Peoples R China
[4] Shanghai Jiao Tong Univ, Sch Biomed Engn, Inst Med Imaging Technol, Shanghai 200030, Peoples R China
[5] Rutgers New Jersey Med Sch, Dept Orthopaed, Newark, NJ 07103 USA
[6] Rutgers State Univ, Dept Comp Sci, Piscataway, NJ 08854 USA
[7] Shanghai Jiao Tong Univ, Sch Med, Shanghai Peoples Hosp 9, Shanghai Key Lab Orthopaed Implants, Shanghai 200011, Peoples R China
[8] Minist Educ, Engn Res Ctr Digital Med & Clin Translat, Shanghai 200240, Peoples R China
[9] Shanghai Jiao Tong Univ, Qing Yuan Res Inst, Shanghai 200240, Peoples R China
关键词
Bone tumor segmentation; Deep learning; Convolutional neural network; Multi-view fusion; Relation-context representation learning; Limb salvage; CONVOLUTIONAL NEURAL-NETWORKS; PATIENT-SPECIFIC INSTRUMENTS; SPARSE REPRESENTATION; IMAGE; RECONSTRUCTION; SEGMENTATION; RESECTION; MRI; CHONDROSARCOMA; OSTEOSARCOMA;
D O I
10.1016/j.media.2020.101954
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Limb salvage surgery of malignant pelvic tumors is the most challenging procedure in musculoskeletal oncology due to the complex anatomy of the pelvic bones and soft tissues. It is crucial to accurately resect the pelvic tumors with appropriate margins in this procedure. However, there is still a lack of efficient and repetitive image planning methods for tumor identification and segmentation in many hos-pitals. In this paper, we present a novel deep learning-based method to accurately segment pelvic bone tumors in MRI. Our method uses a multi-view fusion network to extract pseudo-3D information from two scans in different directions and improves the feature representation by learning a relational context. In this way, it can fully utilize spatial information in thick MRI scans and reduce over-fitting when learning from a small dataset. Our proposed method was evaluated on two independent datasets collected from 90 and 15 patients, respectively. The segmentation accuracy of our method was superior to several com-paring methods and comparable to the expert annotation, while the average time consumed decreased about 100 times from 1820.3 seconds to 19.2 seconds. In addition, we incorporate our method into an efficient workflow to improve the surgical planning process. Our workflow took only 15 minutes to com-plete surgical planning in a phantom study, which is a dramatic acceleration compared with the 2-day time span in a traditional workflow. (c) 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ )
引用
收藏
页数:14
相关论文
共 72 条
  • [1] Three-dimension-printed custom-made prosthetic reconstructions: from revision surgery to oncologic reconstructions
    Angelini, Andrea
    Trovarelli, Giulia
    Berizzi, Antonio
    Pala, Elisa
    Breda, Anna
    Ruggieri, Pietro
    [J]. INTERNATIONAL ORTHOPAEDICS, 2019, 43 (01) : 123 - 132
  • [2] [Anonymous], 2010, ICML
  • [3] Bakas S, 2018, ARXIV PREPRINT ARXIV
  • [4] Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: Application to bone tumor radiographs
    Banerjee, Imon
    Kurtz, Camille
    Devorah, Alon Edward
    Do, Bao
    Rubin, Daniel L.
    Beaulieu, Christopher F.
    [J]. JOURNAL OF BIOMEDICAL INFORMATICS, 2018, 84 : 123 - 135
  • [5] Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer
    Bejnordi, Babak Ehteshami
    Veta, Mitko
    van Diest, Paul Johannes
    van Ginneken, Bram
    Karssemeijer, Nico
    Litjens, Geert
    van der Laak, Jeroen A. W. M.
    [J]. JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2017, 318 (22): : 2199 - 2210
  • [6] Are Custom Triflange Acetabular Components Effective for Reconstruction of Catastrophic Bone Loss?
    Berasi, Carl C.
    Berend, Keith R.
    Adams, Joanne B.
    Ruh, Erin L.
    Lombardi, Adolph V., Jr.
    [J]. CLINICAL ORTHOPAEDICS AND RELATED RESEARCH, 2015, 473 (02) : 528 - 535
  • [7] Improved Accuracy with 3D Planning and Patient-Specific Instruments During Simulated Pelvic Bone Tumor Surgery
    Cartiaux, Olivier
    Paul, Laurent
    Francq, Bernard G.
    Banse, Xavier
    Docquier, Pierre-Louis
    [J]. ANNALS OF BIOMEDICAL ENGINEERING, 2014, 42 (01) : 205 - 213
  • [8] Chen T, 2020, PR MACH LEARN RES, V119
  • [9] Bone Tumor Diagnosis Using a Na⟨ve Bayesian Model of Demographic and Radiographic Features
    Do, Bao H.
    Langlotz, Curtis
    Beaulieu, Christopher F.
    [J]. JOURNAL OF DIGITAL IMAGING, 2017, 30 (05) : 640 - 647
  • [10] Multi-task Self-Supervised Visual Learning
    Doersch, Carl
    Zisserman, Andrew
    [J]. 2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, : 2070 - 2079