Heterogeneous Features Fusion with Collaborative Representation Learning for 3D Action Recognition

Human action recognition of depth sensors has drawn wide attentions in computer vision and multimedia processing areas. In contrast to simple periodic actions, irrelevant actions or sharing sub-actions between different classes of two person non-periodic interactions make this task challenging. This paper presents heterogeneous features fusion with Collaborative Representation (CR) to address this challenge. Two effective high dimensional low-level features are developed from depth image sequence and skeleton pose sequence respectively. In the Canonical Correlations Analysis (CCA) feature space of these two features, Collaborative Representation (CR) is learned and adopted as the final high-level discriminative representation. Experiments on two depth action datasets (SBU Kinect-Interaction and MSR Action 3D) show that the proposed method is superior to the state-of-the-art methods compared, including some recent deep learning based methods.