Fully Attentional Network for Skeleton-Based Action Recognition

With the extraordinary ability by representing human body structure as a spatial graph, graph convolution networks (GCNs) have progressed much in skeleton-based action recognition. However, these methods usually use a predefined graph to represent human body structure, which is limited to 1-hop neighborhood with fixed weights. To handle these limitations, we propose a fully attentional network (FAN). It dynamically computes the edge weights for each input sample between graph nodes, thus avoiding the predefined fixed weights. Besides, it could attend to distant nodes by calculating their edge weights based on their similarities, thus avoiding the limited spatial receptive field. As an effective feature extractor, FAN achieves new state-of-the-art accuracy on three large-scale datasets, i.e., NTU RGB+D 60, NTU RGB+D 120 and Kinetics Skeleton 400. Visualizations are given to verify that FAN could dynamically emphasize the graph nodes that are important in expressing an action.