Extracting Discriminative Parts with Flexible Number from Low-Rank Features for Human Action Recognition

In this paper, we make full use of the complementarity between low-rank feature and part-based feature and present a novel method extracting discriminative parts with flexible number from low-rank features for action recognition. The proposed method avoids some intermediate processing steps (e.g., actor segmentation, body tracking) required by many traditional methods and can greatly avoid memorizing background information suffered by traditional part-based methods. In addition, traditional part-based methods usually set a fixed and identical number of discriminative parts for all action categories neglecting the differences of recognizing complexity among different action categories. On the contrary, we automatically extract discriminative parts with flexible number for each action category by introducing group sparse regularizer into our model, which is more reasonable and effective. In our method, we first extract low-rank features of all action sequences and transform them into corresponding low-rank images. Then, we densely sample each low-rank image into a large number of parts in multi-scale and represent each part into a feature vector. Afterward, our model automatically learn a set of discriminative part detectors with flexible number for each action category. We further define new similarity constraints to force the responses of detected parts from the same class more similar and consistent and that from different class more different. Finally, we define a corresponding recognition criterion to perform final action recognition. The efficacy of the proposed method is verified on three public datasets, and experimental results have shown the promising results of our method for human action recognition.