Human Fall Detection Using 3D Multi-Stream Convolutional Neural Networks with Fusion

被引:21
作者
Alanazi, Thamer [1 ]
Muhammad, Ghulam [1 ]
机构
[1] King Saud Univ, Coll Comp & Informat Sci, Dept Comp Engn, Riyadh 11543, Saudi Arabia
关键词
human fall detection; deep learning; convolution neural networks; fusion networks; 3D-CNN and 2D-CNN; GESTURE RECOGNITION; DETECTION SYSTEM; REPRESENTATIONS; CNN;
D O I
10.3390/diagnostics12123060
中图分类号
R5 [内科学];
学科分类号
1002 ; 100201 ;
摘要
Human falls, especially for elderly people, can cause serious injuries that might lead to permanent disability. Approximately 20-30% of the aged people in the United States who experienced fall accidents suffer from head trauma, injuries, or bruises. Fall detection is becoming an important public healthcare problem. Timely and accurate fall incident detection could enable the instant delivery of medical services to the injured. New advances in vision-based technologies, including deep learning, have shown significant results in action recognition, where some focus on the detection of fall actions. In this paper, we propose an automatic human fall detection system using multi-stream convolutional neural networks with fusion. The system is based on a multi-level image-fusion approach of every 16 frames of an input video to highlight movement differences within this range. This results of four consecutive preprocessed images are fed to a new proposed and efficient lightweight multi-stream CNN model that is based on a four-branch architecture (4S-3DCNN) that classifies whether there is an incident of a human fall. The evaluation included the use of more than 6392 generated sequences from the Le2i fall detection dataset, which is a publicly available fall video dataset. The proposed method, using three-fold cross-validation to validate generalization and susceptibility to overfitting, achieved a 99.03%, 99.00%, 99.68%, and 99.00% accuracy, sensitivity, specificity, and precision, respectively. The experimental results prove that the proposed model outperforms state-of-the-art models, including GoogleNet, SqueezeNet, ResNet18, and DarkNet19, for fall incident detection.
引用
收藏
页数:20
相关论文
共 78 条
[1]  
Adhikari K, 2017, PROCEEDINGS OF THE FIFTEENTH IAPR INTERNATIONAL CONFERENCE ON MACHINE VISION APPLICATIONS - MVA2017, P81, DOI 10.23919/MVA.2017.7986795
[2]   Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation [J].
Al-Hammadi, Muneer ;
Muhammad, Ghulam ;
Abdul, Wadood ;
Alsulaiman, Mansour ;
Bencherif, Mohammed A. ;
Alrayes, Tareq S. ;
Mathkour, Hassan ;
Mekhtiche, Mohamed Amine .
IEEE ACCESS, 2020, 8 (08) :192527-192542
[3]   Hand Gesture Recognition for Sign Language Using 3DCNN [J].
Al-Hammadi, Muneer ;
Muhammad, Ghulam ;
Abdul, Wadood ;
Alsulaiman, Mansour ;
Bencherif, Mohamed A. ;
Mekhtiche, Mohamed Amine .
IEEE ACCESS, 2020, 8 :79491-79509
[4]  
Alaoui A.Y., 2017, P 2 INT C BIG DATA C
[5]   Fall Detection of Elderly People Using the Manifold of Positive Semidefinite Matrices [J].
Alaoui, Abdessamad Youssfi ;
Tabii, Youness ;
Thami, Rachid Oulad Haj ;
Daoudi, Mohamed ;
Berretti, Stefano ;
Pala, Pietro .
JOURNAL OF IMAGING, 2021, 7 (07)
[6]  
Ali SF, 2013, 2013 16TH INTERNATIONAL MULTI TOPIC CONFERENCE (INMIC), P101, DOI 10.1109/INMIC.2013.6731332
[7]   A Comprehensive Survey of the Internet of Things (IoT) and AI-Based Smart Healthcare [J].
Alshehri, Fatima ;
Muhammad, Ghulam .
IEEE ACCESS, 2021, 9 (09) :3660-3678
[8]   Deep learning techniques for classification of electroencephalogram (EEG) motor imagery (MI) signals: a review [J].
Altaheri, Hamdi ;
Muhammad, Ghulam ;
Alsulaiman, Mansour ;
Amin, Syed Umar ;
Altuwaijri, Ghadir Ali ;
Abdul, Wadood ;
Bencherif, Mohamed A. ;
Faisal, Mohammed .
NEURAL COMPUTING & APPLICATIONS, 2023, 35 (20) :14681-14722
[9]   A Multi-Branch Convolutional Neural Network with Squeeze-and-Excitation Attention Blocks for EEG-Based Motor Imagery Signals Classification [J].
Altuwaijri, Ghadir Ali ;
Muhammad, Ghulam ;
Altaheri, Hamdi ;
Alsulaiman, Mansour .
DIAGNOSTICS, 2022, 12 (04)
[10]   A Multibranch of Convolutional Neural Network Models for Electroencephalogram-Based Motor Imagery Classification [J].
Altuwaijri, Ghadir Ali ;
Muhammad, Ghulam .
BIOSENSORS-BASEL, 2022, 12 (01)