Spatio-temporal graph-based self-labeling for video anomaly detection

Video anomaly detection (VAD) aims to identify abnormal events in a video sequence. Existing methods achieve VAD by learning the decision boundary between the normal space and the abnormal space pre-defined in the training data. However, these methods trend to neglect the distribution gap between the pre-defined abnormal space and the real one, which lead to overfitting on the normal space or bias toward the pre-defined abnormal space. In this paper, we propose a spatio-temporal graph-based self-labeling method that not only focuses on the pre-defined abnormal space but considers the real abnormal space, enabling it to capture the decision boundary between the normal space and a complementary space, called as the not-normal space. We first construct a spatio-temporal graph (ST-Graph) based on the objects of input video and utilize a spatio-temporal graph convolution network (ST-GCN) to model the interaction between objects. We then propose a self-labeling- based learning mechanism that encourages the proposed ST-GCN to record the normal events while abstaining from labeling the pseudo-abnormal events, thereby aggregating the pre-defined and real abnormal spaces into not-normal space. To evaluate the model performance on localizing anomalous objects and capturing interactions between objects, we further introduce an object-level criterion that bridges frame-level and pixel- level criteria. Our method is validated on three datasets and achieves state-of-the-art frame-level AUC results on Avenue (92.5%), and outperforms existing ST-Graph-based methods on UCSD Ped2 (96.5%) and ShanghaiTech (76.8%).