Multi-Dimensional Dynamic Topology Learning Graph Convolution for Skeleton-Based Action Recognition

Graph convolution is widely used in skeleton-based action recognition because of its effectiveness of processing graph data. However, the existing graph convolution methods use the shared graph topology for feature aggregation on all frames or channels, which greatly limits the representation ability of graph convolution network. In order to solve these problems, a multi-dimensional dynamic topology learning graph convolution is proposed in this paper to dynamically model the topology with temporal and channel specificity. The multi-dimensional dynamic topology learning graph convolution mainly includes three parts: pure joint topology learning graph convolution (J-GC), dynamic temporal-wise topology learning graph convolution (DTW-GC) and channel-wise topology learning graph convolution (CW-GC). In particular, in DTW-GC, a dynamic skeleton topology modeling method (DSTL) is designed to efficiently model the dynamic skeleton topology with rich global spatio-temporal topological features. Finally, by combining multi-dimensional dynamic topology learning graph convolution with multi-scale temporal convolution (Muti-Scale TCN), a graph convolution network with powerful modeling capability is constructed in this paper. In addition, in order to supplement the spatial information of skeleton data, the relative joint data and relative bone data are introduced for multi-stream network fusion. Our method achieves 92.64% and 89.29% accuracy on NTU-RGB+D and NTU-RGB+D 120 datasets, respectively, which is superior to the current state-of-the-art methods. © 2024 Chinese Institute of Electronics. All rights reserved.