Overlooked Video Classification in Weakly Supervised Video Anomaly Detection

Current weakly supervised video anomaly detection algorithms mostly use multiple instance learning (MIL) or their varieties. Almost all recent approaches focus on how to select the correct snippets for training to improve performance. They overlook or do not realize the power of whole-video classification in improving the performance of anomaly detection, particularly on negative videos. In this paper, we study the power of whole-video classification supervision explicitly using a BERT or LSTM. With this BERT or LSTM, CNN features of all snippets of a video can be aggregated into a single feature which can be used for whole-video classification. This simple yet powerful whole-video classification supervision, combined with the MIL and RTFM framework, brings extraordinary performance improvement on all three major video anomaly detection datasets. Particularly it improves the mean average precision (mAP) on the XD-Violence from SOTA 78.84% to new 82.10%. These results demonstrate this video classification can be combined with other anomaly detection algorithms to achieve better performance. The code is publicly available at https://github. com/wjtan99/BERT_Anomaly_Video_Classification.