Multi-hop graph transformer network for 3D human pose estimation

被引:6
作者
Islam, Zaedul [1 ]
Ben Hamza, A. [1 ]
机构
[1] Concordia Univ, Concordia Inst Informat Syst Engn, Montreal, PQ, Canada
基金
加拿大自然科学与工程研究理事会;
关键词
3D human pose estimation; Graph convolutional network; Transformer; Multi-hop; Dilated convolution;
D O I
10.1016/j.jvcir.2024.104174
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Accurate 3D human pose estimation is a challenging task due to occlusion and depth ambiguity. In this paper, we introduce a multi -hop graph transformer network designed for 2D -to -3D human pose estimation in videos by leveraging the strengths of multi-head self-attention and multi -hop graph convolutional networks with disentangled neighborhoods to capture spatio-temporal dependencies and handle long-range interactions. The proposed network architecture consists of a graph attention block composed of stacked layers of multi-head self-attention and graph convolution with learnable adjacency matrix, and a multi -hop graph convolutional block comprised of multi -hop convolutional and dilated convolutional layers. The combination of multi-head self-attention and multi -hop graph convolutional layers enables the model to capture both local and global dependencies, while the integration of dilated convolutional layers enhances the model's ability to handle spatial details required for accurate localization of the human body joints. Extensive experiments demonstrate the effectiveness and generalization ability of our model, achieving competitive performance on benchmark datasets.
引用
收藏
页数:12
相关论文
共 56 条
[1]   3D Human Pose Estimation Using Mobius Graph Convolutional Networks [J].
Azizi, Niloofar ;
Possegger, Horst ;
Rodola, Emanuele ;
Bischof, Horst .
COMPUTER VISION - ECCV 2022, PT I, 2022, 13661 :160-178
[2]  
Cai J., 2023, P IEEE INT C AC SPEE, P1
[3]   Exploiting Spatial-temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks [J].
Cai, Yujun ;
Ge, Liuhao ;
Liu, Jun ;
Cai, Jianfei ;
Cham, Tat-Jen ;
Yuan, Junsong ;
Thalmann, Nadia Magnenat .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :2272-2281
[4]   Anatomy-Aware 3D Human Pose Estimation With Bone-Based Pose Decomposition [J].
Chen, Tianlang ;
Fang, Chen ;
Shen, Xiaohui ;
Zhu, Yiheng ;
Chen, Zhili ;
Luo, Jiebo .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2022, 32 (01) :198-209
[5]   Cascaded Pyramid Network for Multi-Person Pose Estimation [J].
Chen, Yilun ;
Wang, Zhicheng ;
Peng, Yuxiang ;
Zhang, Zhiqiang ;
Yu, Gang ;
Sun, Jian .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :7103-7112
[6]   Towards Part-Aware Monocular 3D Human Pose Estimation: An Architecture Search Approach [J].
Chen, Zerui ;
Huang, Yan ;
Yu, Hongyuan ;
Xue, Bin ;
Han, Ke ;
Guo, Yiru ;
Wang, Liang .
COMPUTER VISION - ECCV 2020, PT III, 2020, 12348 :715-732
[7]   3D Human Pose Estimation=2D Pose Estimation plus Matching [J].
Chen, Ching-Hang ;
Ramanan, Deva .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :5759-5767
[8]  
Dosovitskiy A., 2021, 9 INT C LEARN REPR I
[9]   Home-based Physical Therapy with an Interactive Computer Vision System [J].
Gu, Yiwen ;
Pandit, Shreya ;
Saraee, Elham ;
Nordahl, Timothy ;
Ellis, Terry ;
Betke, Margrit .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW), 2019, :2619-2628
[10]   In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representations [J].
Habibie, Ikhsanul ;
Xu, Weipeng ;
Mehta, Dushyant ;
Pons-Moll, Gerard ;
Theobalt, Christian .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :10897-10906