Human Behavior Understanding in Big Multimedia Data Using CNN based Facial Expression Recognition

被引:52
作者
Sajjad, Muhammad [1 ]
Zahir, Sana [1 ]
Ullah, Amin [2 ]
Akhtar, Zahid [4 ]
Muhammad, Khan [3 ]
机构
[1] Islamia Coll Peshawar, Dept Comp Sci, Digital Image Proc Lab, Peshawar, Pakistan
[2] Sejong Univ, Digital Contents Res Inst, Intelligent Media Lab, Seoul, South Korea
[3] Sejong Univ, Dept Software, Seoul, South Korea
[4] Univ Memphis, Dept Comp Sci, Memphis, TN 38152 USA
关键词
Big multimedia data; Convolutional neural network; Detection and tracking; Facial expression recognition; Human behavior analysis;
D O I
10.1007/s11036-019-01366-9
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Human behavior analysis from big multimedia data has become a trending research area with applications to various domains such as surveillance, medical, sports, and entertainment. Facial expression analysis is one of the most prominent clues to determine the behavior of an individual, however, it is very challenging due to variations in face poses, illuminations, and different facial tones. In this paper, we analyze human behavior using facial expressions by considering some famous TV-series videos. Firstly, we detect faces using Viola-jones algorithm followed by tracking through Kanade-Lucas-Tomasi (KLT) algorithm. Secondly, we use histogram of oriented gradients (HOG) features with support vector machine (SVM) classifier for facial recognition. Next, we recognize facial expressions using the proposed light-weight convolutional neural network (CNN). We utilize data augmentation techniques to overcome the issue of appearance of faces from different views and lightening conditions in video data. Finally, we predict human behaviors using an occurrence matrix acquired from facial recognition and expressions. The subjective and objective experimental evaluations prove better performance for both facial expression recognition and human behavior understanding.
引用
收藏
页码:1611 / 1621
页数:11
相关论文
共 44 条
[1]   5G-enabled devices and smart-spaces in social-IoT: An overview [J].
Al-Turjrnan, Fadi .
FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2019, 92 :732-744
[2]  
[Anonymous], IEEE T IND ELECT
[3]  
[Anonymous], 2004, CVPR WORKSHOP FACE P
[4]  
[Anonymous], 2001, PYRAMIDAL IMPLEMENTA
[5]  
[Anonymous], 2005, INT C COMP VIS PATT
[6]  
Bartlett MS, 2005, PROC CVPR IEEE, P568
[7]   Human behaviour recognition in data-scarce domains [J].
Baxter, Rolf H. ;
Robertson, Neil M. ;
Lane, David M. .
PATTERN RECOGNITION, 2015, 48 (08) :2377-2393
[8]   Facial expressions of emotion (KDEF): Identification under different display-duration conditions [J].
Calvo, Manuel G. ;
Lundqvist, Daniel .
BEHAVIOR RESEARCH METHODS, 2008, 40 (01) :109-115
[9]  
Chang K-Y, 2013, SYST MAN CYB SMC 201
[10]  
Chen J., 2014, INT WORKSH EL COMP E