Graph Convolutional Networks for Skeleton-Based Action Recognition with LSTM using Tool-Information

Skeleton-based action recognition using a Graph Convolutional Network (GCN) achieved remarkable results by reconstructing a person's skeleton into a graph. However, there are fundamental problems with existing GCN-based models. Generally, human action is greatly affected by the tools used, but in traditional GCN models, action recognition is performed without using the information in the tools. For example, a person holding a pen is limited to the act of writing. A person holding a ball is limited to the action of throwing or receiving the ball. In other words, it is an inaccurate method to judge action only by recognizing the movements of bones used in existing methods. Therefore, a graph was made to reflect the information on the tool. We identify tool-information using the LSTM classifier and propose GCNs for skeleton-based action recognition with LSTM using tool-information. Additionally, we apply a new graph construction and utilize the Learnable Adjacency Matrix. The proposed method is applied to the existing model and comparative evaluation was performed between the model with and without the applied algorithm. The evaluations showed consistent performance improvement, and the proposed method applied to the baseline models achieved state-of-the-art performance.