Scale Adaptive Graph Convolutional Network for Skeleton-Based Action Recognition

In skeleton-based action recognition, graph convolutional network(GCN), which models the human skeleton sequences as spatiotemporal graphs, have achieved excellent performance. However, in existing GCN-based methods, the topology of the graph is set manually, and it is fixed over all layers and input samples. This approach may not be optimal for diverse samples. Constructing an scale adaptive graph based on sample characteristics can better capture spatiotemporal features. Moreover, most methods do not explicitly exploit the multiple scales of body components, which carry crucial information for action recognition. In this paper, we proposed a scale adaptive graph convolutional network comprising a dynamic scale graph convolution module and a multiscale fusion module. Specifically, we first used an a priori and attention mechanism to construct an activity judger, which can divide each keypoint into two parts based on whether it is active;thereafter, a scale adaptive graph was automatically learned. This module accelerated the feature transfer between nodes while minimizing the feature loss. Furthermore, we proposed a multiscale fusion module based on the channel attention mechanism to extract features at different scales and fuse features across scales. Moreover, we used a four-stream framework to model the first-order, second-order, and motion information of a skeleton, which shows notable improvement in terms of recognition accuracy. Extensive experiments on the NTU-RGBD dataset demonstrate the effectiveness of our method. Results show that the algorithm achieves 89.7% and 96.1% classification accuracy on the cross-subject(CS) and cross-view(CV) subsets of the NTU-RGBD dataset, respectively, thus significantly improving the accuracy of action recognition. © 2022, Editorial Board of Journal of Tianjin University(Science and Technology). All right reserved.