Automatic landmark detection and mapping for 2D/3D registration with BoneNet

被引:4
|
作者
Nguyen, Van [1 ]
Pereira, Luis F. Alves F. [1 ,2 ]
Liang, Zhihua [1 ]
Mielke, Falk [1 ,3 ]
Van Houtte, Jeroen [1 ]
Sijbers, Jan [1 ]
De Beenhouwer, Jan [1 ]
机构
[1] Univ Antwerp, Dept Phys, Imec Vis Lab, Antwerp, Belgium
[2] Univ Fed Agreste Pernambuco, Dept Ciencia Comp, Garanhuns, Brazil
[3] Univ Antwerp, Dept Biol, Antwerp, Belgium
关键词
2D; 3D registration; landmark-based registration; pose estimation; automatic landmark detection; deep learning; 2D-3D REGISTRATION; IMAGES;
D O I
10.3389/fvets.2022.923449
中图分类号
S85 [动物医学(兽医学)];
学科分类号
0906 ;
摘要
The 3D musculoskeletal motion of animals is of interest for various biological studies and can be derived from X-ray fluoroscopy acquisitions by means of image matching or manual landmark annotation and mapping. While the image matching method requires a robust similarity measure (intensity-based) or an expensive computation (tomographic reconstruction-based), the manual annotation method depends on the experience of operators. In this paper, we tackle these challenges by a strategic approach that consists of two building blocks: an automated 3D landmark extraction technique and a deep neural network for 2D landmarks detection. For 3D landmark extraction, we propose a technique based on the shortest voxel coordinate variance to extract the 3D landmarks from the 3D tomographic reconstruction of an object. For 2D landmark detection, we propose a customized ResNet18-based neural network, BoneNet, to automatically detect geometrical landmarks on X-ray fluoroscopy images. With a deeper network architecture in comparison to the original ResNet18 model, BoneNet can extract and propagate feature vectors for accurate 2D landmark inference. The 3D poses of the animal are then reconstructed by aligning the extracted 2D landmarks from X-ray radiographs and the corresponding 3D landmarks in a 3D object reference model. Our proposed method is validated on X-ray images, simulated from a real piglet hindlimb 3D computed tomography scan and does not require manual annotation of landmark positions. The simulation results show that BoneNet is able to accurately detect the 2D landmarks in simulated, noisy 2D X-ray images, resulting in promising rigid and articulated parameter estimations.
引用
收藏
页数:17
相关论文
共 50 条
  • [41] Fundamental limits in 3D landmark localization
    Rohr, K
    INFORMATION PROCESSING IN MEDICAL IMAGING, PROCEEDINGS, 2005, 3565 : 286 - 298
  • [42] Comparison and ensemble of 2D and 3D approaches for COVID-19 detection in CT images
    Ahmed, Sara Atito Ali
    Yavuz, Mehmet Can
    Sen, Mehmet Umut
    Gulsen, Faith
    Tutar, Onur
    Korkmazer, Bora
    Samanci, Cesur
    Sirolu, Sabri
    Hamid, Rauf
    Eryurekli, Ali Ergun
    Mammadov, Toghrul
    Yanikoglu, Berrin
    NEUROCOMPUTING, 2022, 488 : 457 - 469
  • [43] Early Detection of Macular Atrophy Automated Through 2D and 3D Unet Deep Learning
    Wei, Wei
    Patel, Radhika Pooja
    Laponogov, Ivan
    Cordeiro, Maria Francesca
    Veselkov, Kirill
    BIOENGINEERING-BASEL, 2024, 11 (12):
  • [44] 3D cephalometric landmark detection by multiple stage deep reinforcement learning
    Kang, Sung Ho
    Jeon, Kiwan
    Kang, Sang-Hoon
    Lee, Sang-Hwy
    SCIENTIFIC REPORTS, 2021, 11 (01)
  • [45] Face 2D to 3D Reconstruction Network Based on Head Pose and 3D Facial Landmarks
    Xu, Yuanquan
    Jung, Cheolkon
    2021 INTERNATIONAL CONFERENCE ON VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP), 2021,
  • [46] Registration of 3D shapes under anisotropic scaling
    Chen, Elvis C. S.
    McLeod, A. Jonathan
    Baxter, John S. H.
    Peters, Terry M.
    INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY, 2015, 10 (06) : 867 - 878
  • [47] 2D Object Detection: A Survey
    Malagoli, Emanuele
    Di Persio, Luca
    MATHEMATICS, 2025, 13 (06)
  • [48] Image-to-Graph Convolutional Network for 2D/3D Deformable Model Registration of Low-Contrast Organs
    Nakao, Megumi
    Nakamura, Mitsuhiro
    Matsuda, Tetsuya
    IEEE TRANSACTIONS ON MEDICAL IMAGING, 2022, 41 (12) : 3747 - 3761
  • [49] CAR-Net: A Deep Learning-Based Deformation Model for 3D/2D Coronary Artery Registration
    Wu, Wei
    Zhang, Jingyang
    Peng, Wenjia
    Xie, Hongzhi
    Zhang, Shuyang
    Gu, Lixu
    IEEE TRANSACTIONS ON MEDICAL IMAGING, 2022, 41 (10) : 2715 - 2727
  • [50] 2D Registration Guided Models for Semi-automatic MRI Prostate Segmentation
    Cheng, Ruida
    Turkbey, Baris
    Senseney, Justin
    Bernardo, Marcelino
    Bokinsky, Alexandra
    Gandler, William
    McCreedy, Evan
    Pohida, Thomas
    Choyke, Peter
    McAuliffe, Matthew J.
    MEDICAL IMAGING 2013: IMAGE PROCESSING, 2013, 8669