Multidimensional Prototype Refactor Enhanced Network for Few-Shot Action Recognition

Few-shot action recognition classifies new actions with only few training samples, of which the mainstream methods adopt class means to obtain prototypes as the representations of each category. However, affected by sample capacity and extreme samples, mean-of-class prototypes can't well represent the average level of samples. In this paper, we enhance the prototypes from multiple dimensions for better classification. We firstly propose a novel similarity optimization mechanism where Prototype Aggregation Adaptive Loss (PAAL) is designed to deeply mine the similarity between samples and prototypes for enhancing the ability of inter-class differential detail identification. Secondly, for mitigating the impact of the samples on class prototypes, we refactor the prototype calculation formula with Cross-Enhanced Prototype (CEP) to narrow intra-class differences in which Reweighted Similarity Attention (RSA) is designed to update prototypes. Finally, Dynamic Temporal Transformation (DTT) is proposed to alleviate inconsistent distribution of temporal information for obtaining better video-level descriptors. Extensive experiments on standard benchmark datasets demonstrate that our proposed method achieves the state-of-the-art results.