Automatic landmark detection and mapping for 2D/3D registration with BoneNet

被引:4
|
作者
Nguyen, Van [1 ]
Pereira, Luis F. Alves F. [1 ,2 ]
Liang, Zhihua [1 ]
Mielke, Falk [1 ,3 ]
Van Houtte, Jeroen [1 ]
Sijbers, Jan [1 ]
De Beenhouwer, Jan [1 ]
机构
[1] Univ Antwerp, Dept Phys, Imec Vis Lab, Antwerp, Belgium
[2] Univ Fed Agreste Pernambuco, Dept Ciencia Comp, Garanhuns, Brazil
[3] Univ Antwerp, Dept Biol, Antwerp, Belgium
关键词
2D; 3D registration; landmark-based registration; pose estimation; automatic landmark detection; deep learning; 2D-3D REGISTRATION; IMAGES;
D O I
10.3389/fvets.2022.923449
中图分类号
S85 [动物医学(兽医学)];
学科分类号
0906 ;
摘要
The 3D musculoskeletal motion of animals is of interest for various biological studies and can be derived from X-ray fluoroscopy acquisitions by means of image matching or manual landmark annotation and mapping. While the image matching method requires a robust similarity measure (intensity-based) or an expensive computation (tomographic reconstruction-based), the manual annotation method depends on the experience of operators. In this paper, we tackle these challenges by a strategic approach that consists of two building blocks: an automated 3D landmark extraction technique and a deep neural network for 2D landmarks detection. For 3D landmark extraction, we propose a technique based on the shortest voxel coordinate variance to extract the 3D landmarks from the 3D tomographic reconstruction of an object. For 2D landmark detection, we propose a customized ResNet18-based neural network, BoneNet, to automatically detect geometrical landmarks on X-ray fluoroscopy images. With a deeper network architecture in comparison to the original ResNet18 model, BoneNet can extract and propagate feature vectors for accurate 2D landmark inference. The 3D poses of the animal are then reconstructed by aligning the extracted 2D landmarks from X-ray radiographs and the corresponding 3D landmarks in a 3D object reference model. Our proposed method is validated on X-ray images, simulated from a real piglet hindlimb 3D computed tomography scan and does not require manual annotation of landmark positions. The simulation results show that BoneNet is able to accurately detect the 2D landmarks in simulated, noisy 2D X-ray images, resulting in promising rigid and articulated parameter estimations.
引用
收藏
页数:17
相关论文
共 50 条
  • [1] Fully automatic 3D/2D subtracted angiography registration
    Kerrien, E
    Berger, MO
    Maurincomme, E
    Launay, L
    Vaillant, R
    Picard, L
    MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION, MICCAI'99, PROCEEDINGS, 1999, 1679 : 664 - 671
  • [2] Fully automatic 3D/2D subtracted angiography registration
    Kerrien, E
    Launay, L
    Berger, MO
    Maurincomme, E
    Vaillant, R
    Picard, L
    CARS '99: COMPUTER ASSISTED RADIOLOGY AND SURGERY, 1999, 1191 : 989 - 989
  • [3] Computation of DRRs using 3D texture mapping for 2D/3D registration
    Kim, SM
    Kim, YS
    CARS 2005: Computer Assisted Radiology and Surgery, 2005, 1281 : 1313 - 1313
  • [4] Automatic 3D to 2D registration for the photorealistic rendering of urban scenes
    Liu, LY
    Stamos, I
    2005 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOL 2, PROCEEDINGS, 2005, : 137 - 143
  • [5] AUTOMATIC LANDMARK DETECTION FOR 3D URBAN MODELS
    Ganitseva, J.
    Coors, V.
    5TH INTERNATIONAL CONFERENCE ON 3D GEOINFORMATION, 2010, 38-4 (W15): : 37 - 43
  • [6] Automatic 3D facial segmentation and landmark detection
    Segundo, Mauricio P.
    Queirolo, Chaua
    Bellon, Olga R. P.
    Silva, Luciano
    14TH INTERNATIONAL CONFERENCE ON IMAGE ANALYSIS AND PROCESSING, PROCEEDINGS, 2007, : 431 - +
  • [7] An Automatic Multimodal Data Registration Strategy for 2D/3D Information Fusion
    Schierl, Jonathan
    Asari, Vijayan
    Singer, Nina
    Aspiras, Theus
    Stokes, Andrew
    Keaffaber, Brett
    Van Rynbach, Andre
    Decker, Kevin
    Rabb, David
    MULTIMODAL IMAGE EXPLOITATION AND LEARNING 2022, 2022, 12100
  • [8] Efficient automatic 2D/3D registration of cardiac ultrasound and CT images
    Scott, Katy
    Stuart, Duncan
    Peoples, Jacob J.
    Bisleri, Gianluigi
    Ellis, Randy E.
    COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING-IMAGING AND VISUALIZATION, 2021, 9 (04): : 438 - 446
  • [9] Infinite 3D Landmarks: Improving Continuous 2D Facial Landmark Detection
    Chandran, P.
    Zoss, G.
    Gotardo, P.
    Bradley, D.
    COMPUTER GRAPHICS FORUM, 2024, 43 (06)
  • [10] 2D/3D registration of multiple bones
    Ma, Burton
    Stewart, James
    Pichora, David
    Ellis, Randy
    Abolmaesumi, Purang
    2007 ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY, VOLS 1-16, 2007, : 860 - 863