Real-time action detection in video surveillance using a sub-action descriptor with multi-convolutional neural networks

When we say a person is texting, can you tell the person is walking or sitting? Emphatically, no. In order to solve this incomplete representation problem, this paper presents a sub-action descriptor for detailed action detection. The sub-action descriptor consists of three levels: posture, locomotion, and gestures. The three levels provide three sub-action categories for a single action in order to address the representation problem. The proposed action detection model simultaneously localizes and recognizes the actions of multiple individuals in video surveillance using appearance-based temporal features with multi-convolutional neural networks. The proposed approach achieved a mean average precision of 76.6% for frame-based measurement and 83.5% for video-based measurement of the ICVL video surveillance dataset. Extensive experiments on the benchmark KTH dataset demonstrate that the proposed approach achieved better performance, which in turn improves action recognition performance in comparison to the stateof- the-art methods. The action detection model can run at around 25 fps with the ICVL dataset and at more than 80 fps with the KTH dataset, which is suitable for real-time surveillance applications. © ICROS 2018.