Multi-channel network: Constructing efficient GCN baselines for skeleton-based action recognition

被引:7
作者
Hou, Ruijie [1 ]
Wang, Zhihao [1 ]
Ren, Ruimin [2 ]
Cao, Yang [2 ]
Wang, Zhao [1 ,3 ]
机构
[1] Zhejiang Univ, Hangzhou, Peoples R China
[2] Nanjing Normal Univ, Nanjing, Peoples R China
[3] Zhejiang Univ, Ningbo Innovat Ctr, Hangzhou, Peoples R China
来源
COMPUTERS & GRAPHICS-UK | 2023年 / 110卷
基金
中国国家自然科学基金;
关键词
Global and local features; Skeleton based action recognition; Feature fusion; Multi-modality;
D O I
10.1016/j.cag.2022.12.008
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Skeleton-based action sequences are widely used for human behaviour understanding due to their compact characteristics. Most existing work designed Graph Convolutional Networks and integrated multiple input channels rather than the original motion sequence to improve the final performance. However, few of them have been reported on the detailed effects of such multiple input channels. In contrast to them, we systemically study the impact of different input channels and construct a more efficient GCN framework. We have identified the complementary effect between the local frame channel and global sequence channel, which is essential to improve the action recognition accuracy. By coupling local frame and global sequence information with a classical spatial-temporal graph neural network, e.g. MS-G3D, it achieves competitive performance compared with SOTA methods on challeng-ing benchmark datasets. Related code would be available at https://github.com/movearbitrarily/multi-stream.(c) 2022 Elsevier Ltd. All rights reserved.
引用
收藏
页码:111 / 117
页数:7
相关论文
共 43 条
[1]  
[Anonymous], 2013, Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI '13
[2]   Pedestrian Models for Autonomous Driving Part I: Low-Level Models, From Sensing to Tracking [J].
Camara, Fanta ;
Bellotto, Nicola ;
Cosar, Serhan ;
Nathanael, Dimitris ;
Althoff, Matthias ;
Wu, Jingyuan ;
Ruenz, Johannes ;
Dietrich, Andre ;
Fox, Charles W. .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2021, 22 (10) :6131-6151
[3]   Learning a 3D Human Pose Distance Metric from Geometric Pose Descriptor [J].
Chen, Cheng ;
Zhuang, Yueting ;
Nie, Feiping ;
Yang, Yi ;
Wu, Fei ;
Xiao, Jun .
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 2011, 17 (11) :1676-1689
[4]  
Chen Y., 2021, Proceedings of the IEEE/CVF International Conference on Computer Vision, P13359
[5]   Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition [J].
Chen, Tailin ;
Zhou, Desen ;
Wang, Jian ;
Wang, Shidong ;
Guan, Yu ;
He, Xuming ;
Ding, Errui .
PROCEEDINGS OF THE 29TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2021, 2021, :4334-4342
[6]   Skeleton-Based Action Recognition with Shift Graph Convolutional Network [J].
Cheng, Ke ;
Zhang, Yifan ;
He, Xiangyu ;
Chen, Weihan ;
Cheng, Jian ;
Lu, Hanqing .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :180-189
[7]  
Du Y, 2015, PROCEEDINGS 3RD IAPR ASIAN CONFERENCE ON PATTERN RECOGNITION ACPR 2015, P579, DOI 10.1109/ACPR.2015.7486569
[8]  
Du Y, 2015, PROC CVPR IEEE, P1110, DOI 10.1109/CVPR.2015.7298714
[9]   Human action recognition based on low- and high-level data from wearable inertial sensors [J].
Hussein Lopez-Nava, Irvin ;
Munoz-Melendez, Angelica .
INTERNATIONAL JOURNAL OF DISTRIBUTED SENSOR NETWORKS, 2019, 15 (12)
[10]   A New Representation of Skeleton Sequences for 3D Action Recognition [J].
Ke, Qiuhong ;
Bennamoun, Mohammed ;
An, Senjian ;
Sohel, Ferdous ;
Boussaid, Farid .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :4570-4579