Hybrid Classifiers for Spatio-Temporal Abnormal Behavior Detection, Tracking, and Recognition in Massive Hajj Crowds

被引:17
作者
Alafif, Tarik [1 ]
Hadi, Anas [2 ]
Allahyani, Manal [2 ]
Alzahrani, Bander [2 ]
Alhothali, Areej [2 ]
Alotaibi, Reem [2 ]
Barnawi, Ahmed [2 ]
机构
[1] Umm Al Qura Univ, Jamoum Univ Coll, Dept Comp Sci, Mecca 25375, Saudi Arabia
[2] King Abdulaziz Univ, Fac Comp & Informat Technol, Jeddah 21589, Saudi Arabia
关键词
abnormal behaviors; small-scale crowd; large-scale crowd; convolutional neural network; random forest; detection; tracking; recognition; ANOMALY DETECTION; NEURAL-NETWORKS; EVENT DETECTION; LOCALIZATION; SCENES;
D O I
10.3390/electronics12051165
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Individual abnormal behaviors vary depending on crowd sizes, contexts, and scenes. Challenges such as partial occlusions, blurring, a large number of abnormal behaviors, and camera viewing occur in large-scale crowds when detecting, tracking, and recognizing individuals with abnormalities. In this paper, our contribution is two-fold. First, we introduce an annotated and labeled large-scale crowd abnormal behavior Hajj dataset, HAJJv2. Second, we propose two methods of hybrid convolutional neural networks (CNNs) and random forests (RFs) to detect and recognize spatio-temporal abnormal behaviors in small and large-scale crowd videos. In small-scale crowd videos, a ResNet-50 pre-trained CNN model is fine-tuned to verify whether every frame is normal or abnormal in the spatial domain. If anomalous behaviors are observed, a motion-based individual detection method based on the magnitudes and orientations of Horn-Schunck optical flow is proposed to locate and track individuals with abnormal behaviors. A Kalman filter is employed in large-scale crowd videos to predict and track the detected individuals in the subsequent frames. Then, means and variances as statistical features are computed and fed to the RF classifier to classify individuals with abnormal behaviors in the temporal domain. In large-scale crowds, we fine-tune the ResNet-50 model using a YOLOv2 object detection technique to detect individuals with abnormal behaviors in the spatial domain. The proposed method achieves 99.76% and 93.71% of average area under the curves (AUCs) on two public benchmark small-scale crowd datasets, UMN and UCSD, respectively, while the large-scale crowd method achieves 76.08% average AUC using the HAJJv2 dataset. Our method outperforms state-of-the-art methods using the small-scale crowd datasets with a margin of 1.66%, 6.06%, and 2.85% on UMN, UCSD Ped1, and UCSD Ped2, respectively. It also produces an acceptable result in large-scale crowds.
引用
收藏
页数:19
相关论文
共 53 条
[1]   Generative adversarial network based abnormal behavior detection in massive crowd videos: a Hajj case study [J].
Alafif, Tarik ;
Alzahrani, Bander ;
Cao, Yong ;
Alotaibi, Reem ;
Barnawi, Ahmed ;
Chen, Min .
JOURNAL OF AMBIENT INTELLIGENCE AND HUMANIZED COMPUTING, 2021, 13 (8) :4077-4088
[2]  
Alqaysi HH, 2013, IEEE APP IMG PAT
[3]  
[Anonymous], 2001, P SIGGRAPH COURS
[4]  
Bansod SD, 2019, Revised Selected Papers, Part II, P117
[5]   Crowd anomaly detection and localization using histogram of magnitude and momentum [J].
Bansod, Suprit D. ;
Nandedkar, Abhijeet V. .
VISUAL COMPUTER, 2020, 36 (03) :609-620
[6]   Realtime Anomaly Detection using Trajectory-level Crowd Behavior Learning [J].
Bera, Aniket ;
Kim, Sujeong ;
Manocha, Dinesh .
PROCEEDINGS OF 29TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, (CVPRW 2016), 2016, :1289-1296
[7]  
Bottou L., 1991, Proceedings of Neuro-Nimes, Vvol 91, pp 12
[8]   Random forests [J].
Breiman, L .
MACHINE LEARNING, 2001, 45 (01) :5-32
[9]   Social network model for crowd anomaly detection and localization [J].
Chaker, Rima ;
Al Aghbari, Zaher ;
Junejo, Imran N. .
PATTERN RECOGNITION, 2017, 61 :266-281
[10]   PCANet: A Simple Deep Learning Baseline for Image Classification? [J].
Chan, Tsung-Han ;
Jia, Kui ;
Gao, Shenghua ;
Lu, Jiwen ;
Zeng, Zinan ;
Ma, Yi .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2015, 24 (12) :5017-5032