Motion-aware future frame prediction for video anomaly detection based on saliency perception

The anomaly in videos can be considered as a deviation from regular video sequences. Most existing approaches neglect the imbalanced information distribution between the foreground and the background during the process of reconstruction or prediction. To address this problem, we propose a motion-aware future frame prediction network consisting of a frame prediction branch and a saliency perception branch. In particular, the saliency perception branch is designed to predict the most salient targets in the video frame, and the frame prediction branch is used to predict the RGB future frame with the guidance of saliency perception. Besides, a motion-aware attention module is bridged in the frame prediction branch to improve the representation ability of moving targets. Furthermore, a saliency prediction loss and a saliency-guided appearance loss are designed to optimize saliency prediction frames and constrain the weight of foreground. Experiments on three challenging benchmarks demonstrate our competitive performance with the state-of-the-art approaches.