Spatial Graph Convolutional and Temporal Involution Network for Skeleton-based Action Recognition

The existing skeleton-based action recognition methods, based on graph convolutional networks (GCN), cannot well capture continuous behavior information in the time dimension. As traditional Convolution neural network is difficult to extract timing information and capture long-distance relationship. Relying on stacking a large number of convolution kernels to increase the receptive field and feature diversity, not only increases the amounts of parameters and computational complexity, but also causes a large amount of redundancy in the channel dimensions of the convolution kernel. Therefore, we propose the Spatial Graph Convolutional and Temporal involution network (ST-TI). Firstly, in the spatial dimension, GCN is used to obtain the spatial correlation of a single frame of human skeleton points. Then, in the temporal dimension. Involution operation is used to extract the correlations of skeleton points in different frames. The entire model is composed of 9 layers of SG-TI units. Each SG-TI unit uses residual connection and then uses a fully connected layer to ensure that the output and prediction categories have the same dimensionality. Finally, we feed the output feature to a SoftMax classifier. The effect of the model is verified on two public behavior recognition data sets of kinetics and NTU RGB+D. Experiments show that compared with the benchmark network ST-GCN, the algorithm has reduced the number of parameters by 3.7 times and improved the recognition accuracy by 3%