Learning Self-Correlation in Space and Time as Motion Representation for Action Recognition

被引:0
作者
Zhang, Yi [1 ]
Li, Yuchang [1 ]
Liu, Mingwei [1 ]
机构
[1] Sichuan Univ, Dept Comp Sci, Chengdu 610000, Peoples R China
关键词
Action recognition; skeleton joint; spatial-temporal dimension; self-correlation;
D O I
10.1109/LSP.2023.3333210
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Human skeleton, as a compact representation of human action, has received increasing attention in recent years due to the easy accessibility of the human skeleton data. Many previous works capture local physical dependencies among joints, which may miss implicit joint correlations. In this light, we propose a novel motion representation scheme to infer action categories, which includes self-correlation transformation, feature extraction and feature fusion modules. Our network takes a sequence of 2D skeletal heat map as the input, and captures the correlation across frames and adjacent skeleton key points to reason key motion features. Extensive experiments have been conducted on 3 mainstream skeleton-based action recognition datasets, in which our network exhibits superior performances than other state-of-the-art methods.
引用
收藏
页码:1747 / 1751
页数:5
相关论文
共 28 条
[1]   Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset [J].
Carreira, Joao ;
Zisserman, Andrew .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :4724-4733
[2]   Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition [J].
Chen, Yuxin ;
Zhang, Ziqi ;
Yuan, Chunfeng ;
Li, Bing ;
Deng, Ying ;
Hu, Weiming .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :13339-13348
[3]   Skeleton-Based Action Recognition with Shift Graph Convolutional Network [J].
Cheng, Ke ;
Zhang, Yifan ;
He, Xiangyu ;
Chen, Weihan ;
Cheng, Jian ;
Lu, Hanqing .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :180-189
[4]   InfoGCN: Representation Learning for Human Skeleton-based Action Recognition [J].
Chi, Hyung-gun ;
Ha, Myoung Hoon ;
Chi, Seunggeun ;
Lee, Sang Wan ;
Huang, Qixing ;
Ramani, Karthik .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :20154-20164
[5]   PoTion: Pose MoTion Representation for Action Recognition [J].
Choutas, Vasileios ;
Weinzaepfel, Philippe ;
Revaud, Jerome ;
Schmid, Cordelia .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :7024-7033
[6]   Revisiting Skeleton-based Action Recognition [J].
Duan, Haodong ;
Zhao, Yue ;
Chen, Kai ;
Lin, Dahua ;
Dai, Bo .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :2959-2968
[7]   Skeleton-Based Action Recognition With Focusing-Diffusion Graph Convolutional Networks [J].
Gao, Jialin ;
He, Tong ;
Zhou, Xi ;
Ge, Shiming .
IEEE SIGNAL PROCESSING LETTERS, 2021, 28 :2058-2062
[8]   Quo Vadis, Skeleton Action Recognition? [J].
Gupta, Pranay ;
Thatipelli, Anirudh ;
Aggarwal, Aditya ;
Maheshwari, Shubh ;
Trivedi, Neel ;
Das, Sourav ;
Sarvadevabhatla, Ravi Kiran .
INTERNATIONAL JOURNAL OF COMPUTER VISION, 2021, 129 (07) :2097-2112
[9]   MTT: Multi-Scale Temporal Transformer for Skeleton-Based Action Recognition [J].
Kong, Jun ;
Bian, Yuhang ;
Jiang, Min .
IEEE SIGNAL PROCESSING LETTERS, 2022, 29 :528-532
[10]   Improved Shift Graph Convolutional Network for Action Recognition With Skeleton [J].
Li, Chuankun ;
Li, Shuai ;
Gao, Yanbo ;
Guo, Lina ;
Li, Wanqing .
IEEE SIGNAL PROCESSING LETTERS, 2023, 30 :438-442