Multi-feature fusion network for facial expression recognition in the wild

被引:6
作者
Gong, Weijun [1 ]
Wang, Chaoqing [2 ]
Jia, Jinlu [2 ]
Qian, Yurong [1 ,2 ,3 ]
Fan, Yingying [1 ]
机构
[1] Xinjiang Univ, Coll Informat Sci & Engn, Urumqi, Peoples R China
[2] Xinjiang Univ, Coll Software, Urumqi, Peoples R China
[3] Key Lab Signal Detect & Proc Xinjiang Uygur Auton, Urumqi, Peoples R China
基金
美国国家科学基金会;
关键词
Facial expression recognition; multi-feature fusion; feature extraction; deep learning; DEEP;
D O I
10.3233/JIFS-211021
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Facial expression recognition (FER) has been one of the research focuses in recent years due to its significance in human-computer interactions. However, there are still challenges in the field of FER caused by the diversity and variation of facial expressions in real scenes, the singleness of feature type and the lack of enough discriminant features cannot effectively improve the recognition performance. To solve these problems, we propose a Multi-feature Fusion Network (MFNet) with dual-branch based on deep learning. Firstly, the MFNet uses the pyramid parallel multiscale residual network structure with progressive max-pooling of channel attention to extract multi-level facial features and enhance the discrimination of features; In the meantime, a shallow Gabor convolutional network is designed to enhance the adaptation of learned features to the orientation and scale changes and improve the ability to capture local details features; Finally, the maximum expression features obtained by the above two networks are fused to make more effective expression recognition. Experiments on three public large-scale wild FER datasets (RAF-DB, FERPlus, and AffectNet) show that our MFNet has a superior recognition performance than other recognition methods.
引用
收藏
页码:4999 / 5011
页数:13
相关论文
共 46 条
[1]   Covariance Pooling for Facial Expression Recognition [J].
Acharya, Dinesh ;
Huang, Zhiwu ;
Paudel, Danda Pani ;
Van Gool, Luc .
PROCEEDINGS 2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2018, :480-487
[2]   Emotion Recognition in Speech using Cross-Modal Transfer in the Wild [J].
Albanie, Samuel ;
Nagrani, Arsha ;
Vedaldi, Andrea ;
Zisserman, Andrew .
PROCEEDINGS OF THE 2018 ACM MULTIMEDIA CONFERENCE (MM'18), 2018, :292-301
[3]  
Amos B., 2016, CMU School of Computer Science, V6, P20
[4]  
[Anonymous], 2010, 2010 IEEE COMPUTER S, DOI [10. 1109/CVPRW.2010.5543262, DOI 10.1109/CVPRW.2010.5543262]
[5]   Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution [J].
Barsoum, Emad ;
Zhang, Cha ;
Ferrer, Cristian Canton ;
Zhang, Zhengyou .
ICMI'16: PROCEEDINGS OF THE 18TH ACM INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, 2016, :279-283
[6]   Facial Expression Recognition in Video with Multiple Feature Fusion [J].
Chen, Junkai ;
Chen, Zenghai ;
Chi, Zheru ;
Fu, Hong .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2018, 9 (01) :38-50
[7]   Histograms of oriented gradients for human detection [J].
Dalal, N ;
Triggs, B .
2005 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOL 1, PROCEEDINGS, 2005, :886-893
[8]   Video and Image based Emotion Recognition Challenges in the Wild: EmotiW 2015 [J].
Dhall, Abhinav ;
Murthy, O. V. Ramana ;
Goecke, Roland ;
Joshi, Jyoti ;
Gedeon, Tom .
ICMI'15: PROCEEDINGS OF THE 2015 ACM INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, 2015, :423-426
[9]  
Duta IC, 2020, ARXIV PREPRINT ARXIV
[10]  
Fan Yingruo, 2020, IEEE T AFFECTIVE COM, V2020