Multi-Task Learning for Video Surveillance with Limited Data

Learning from limited data in video surveillance is important for sustainable performance while adapting to new information in a scene over time or adapting to a different scene. In a real-world scene, for an anomaly detection algorithm, all possible nominal patterns and behaviors are not typically available immediately for a single training session. In contrast, labeled nominal data patterns may become available irregularly over a long time horizon, and the anomaly detection algorithm needs to quickly learn such new patterns from limited samples for acceptable performance. Otherwise, it would suffer from frequent false alarms. Additionally, the anomaly detection algorithm needs to continually learn new nominal patterns in multiple training sessions without forgetting the previous knowledge and losing performance. Cross-domain adaptability (i.e., transfer learning to another surveillance scene) is another task where the anomaly detection algorithm has to quickly learn from limited nominal training data to achieve acceptable performance. To overcome these challenges, we design a modular framework and use it to extract semantic embeddings, which we then train on by using deep metric learning. Particularly, we study these three problems (few-shot learning, continual learning, cross-domain adaptability) in a multi-task learning setting. We also compare our proposed framework to existing state-of-the-art approaches using various evaluation metrics. The empirical results indicate that the proposed approach is able to outperform the existing approaches on all three tasks for three benchmark datasets.