Human skeleton pose and spatio-temporal feature-based activity recognition using ST-GCN

被引:8
|
作者
Lovanshi, Mayank [1 ]
Tiwari, Vivek [1 ,2 ]
机构
[1] Int Inst Informat Technol IIIT, Naya Raipur, India
[2] ABV Indian Inst Informat Technol & Management, Gwalior, India
关键词
Activity recognition; Pose estimation; ST-GCN; Spatio-temporal feature; Skeleton joints; SPATIAL-DISTRIBUTION; UNIFIED FRAMEWORK; GRADIENTS; MODEL;
D O I
10.1007/s11042-023-16001-9
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Skeleton-based Human Activity Recognition has recently sparked a lot of attention because skeleton data has proven resistant to changes in lighting, body sizes, dynamic camera perspectives, and complicated backgrounds. The Spatial-Temporal Graph Convolutional Networks (ST-GCN) model has been exposed to study spatial and temporal dependencies effectively from skeleton data. However, efficient use of 3D skeleton in-depth information remains a significant challenge, specifically for human joint motion patterns and linkages information. This study attempts a promising solution through a custom ST-GCN model and skeleton joints for human activity recognition. Special attention was given to spatial & temporal features, which were further fed to the classification model for better pose estimation. A comparative study is presented for activity recognition using large-scale databases such as NTU-RGB-D, Kinetics-Skeleton, and Florence 3D datasets. The Custom ST-GCN model outperforms (Top-1 accuracy) the state-of-the-art method on NTU-RGB-D, Kinetics-Skeleton & Florence 3D dataset with a higher margin by 0.7%, 1.25%, and 1.92%, respectively. Similarly, with Top-5 accuracy, the Custom ST-GCN model offers results hike by 0.5%, 0.73% & 1.52%, respectively. It shows that the presented graph-based topologies capture the changing aspects of a motion-based skeleton sequence better than some of the other approaches.
引用
收藏
页码:12705 / 12730
页数:26
相关论文
共 50 条
  • [21] Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition
    Li, Chaolong
    Cui, Zhen
    Zheng, Wenming
    Xu, Chunyan
    Yang, Jian
    THIRTY-SECOND AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTIETH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / EIGHTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2018, : 3482 - 3489
  • [22] Spatio-temporal Feature based Anomaly Event Recognition
    Jin, Dongliang
    Zhu, Songhao
    PROCEEDINGS OF THE 35TH CHINESE CONTROL CONFERENCE 2016, 2016, : 3814 - 3818
  • [23] Human Activity Recognition Based on Transfer Learning with Spatio-Temporal Representations
    Zebhi, Saeedeh
    Almodarresi, S. M. T.
    Abootalebi, Vahid
    INTERNATIONAL ARAB JOURNAL OF INFORMATION TECHNOLOGY, 2021, 18 (06) : 839 - 845
  • [24] Human Action Recognition Using Spatio-Temporal Multiplier Network and Attentive Correlated Temporal Feature
    Indhumathi, C.
    Murugan, V
    Muthulakshmii, G.
    INTERNATIONAL JOURNAL OF IMAGE AND GRAPHICS, 2022, 22 (05)
  • [25] A Hierarchical Spatio-Temporal Model for Human Activity Recognition
    Xu, Wanru
    Miao, Zhenjiang
    Zhang, Xiao-Ping
    Tian, Yi
    IEEE TRANSACTIONS ON MULTIMEDIA, 2017, 19 (07) : 1494 - 1509
  • [26] Spatio-temporal stacking model for skeleton-based action recognition
    Zhong, Yufeng
    Yan, Qiuyan
    APPLIED INTELLIGENCE, 2022, 52 (11) : 12116 - 12130
  • [27] Leveraging Spatio-Temporal Dependency for Skeleton-Based Action Recognition
    Lee, Jungho
    Lee, Minhyeok
    Cho, Suhwan
    Woo, Sungmin
    Jang, Sungjun
    Lee, Sangyoun
    arXiv, 2022,
  • [28] Spatio-temporal segments attention for skeleton-based action recognition
    Qiu, Helei
    Hou, Biao
    Ren, Bo
    Zhang, Xiaohua
    NEUROCOMPUTING, 2023, 518 : 30 - 38
  • [29] Spatio-temporal stacking model for skeleton-based action recognition
    Yufeng Zhong
    Qiuyan Yan
    Applied Intelligence, 2022, 52 : 12116 - 12130
  • [30] Leveraging Spatio-Temporal Dependency for Skeleton-Based Action Recognition
    Lee, Jungho
    Lee, Minhyeok
    Cho, Suhwan
    Woo, Sungmin
    Jang, Sungjun
    Lee, Sangyoun
    2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, : 10221 - 10230