Self-relation attention networks for weakly supervised few-shot activity recognition

Traditional sensor-based human activity recognition (HAR), which requires a large amount of high quality data of a new class for training, can hardly classify new-class activities given only a few labeled samples. Using these models to customize and recognize personalized (new class) activity for a better user experience is a great challenge. It is time-consuming work for users to label abundant new-class time series data. Additionally, these activity sequences are weakly-labeled with existing annotation technology. The purpose of this work is to learn a meta-classifier that generalizes well in weakly-labeled new-class activities even given few noisy samples. On this basis, we combine the power of few-shot learning and weakly supervised classification to achieve the above goal. We propose a novel self-relation attention Prototypical Networks for personalized HAR to extract more discriminative target features from the spatiotemporal features of noisy sensor data. Attention-based meta-learning generates each subsequence contribution to the object activity features automatically on the subsequence-level relation mining in one weakly labeled sample. Therefore, the highlighted object activity features are less affected by the noisy subsequences. Furthermore, we design an augmentation operation for the highlighted features to avoid the biased target activity prototype in each class. Finally, we evaluate the proposed models in extensive experiments, including accuracy evaluations, ablation studies and the influence of thresholds. The results show that our proposal achieves a better performance in weakly-supervised HAR compared with other models. & COPY; 2023 Elsevier B.V. All rights reserved.