Asymmetric information-regularized learning for skeleton-based action recognition

Skeleton-based action recognition has recently achieved remarkable progress, which is typically formulated as a spatial-temporal graph-based classification problem. Nevertheless, most existing approaches straightforwardly model the skeleton topology via a pure encoder and lack explicit guidance to promote the representation capability. To handle the above constraint, the proposed Asymmetric Information-Regularized Graph Convolutional Network (AIR-GCN) explores an effective asymmetric paradigm based on information theory, to force the encoder to learn more representative features. Furthermore, each sample indeed has a unique spatial-temporal topology due to the dynamic action process and AIR-GCN introduces two novel operators to learn spatial-temporal representation beyond the inherent structural relations: leveraging the Topology-regularized Spatial Routing (TrSR) to encode instance-dependent relational graphs and the Topology-regularized Temporal Routing (TrTR) to capture action-specific motion patterns for reducing the ambiguity of highly similar actions. Extensive experiments are conducted on four widely used datasets: Northwestern-UCLA, NTU RGB+D 60, NTU RGB+D 120 and Kinetics Skeleton. The results demonstrate that AIR-GCN achieves notably better performance compared with the state-of-the-art methods.