Recognizing Spontaneous Micro-Expression Using a Three-Stream Convolutional Neural Network

被引：73

作者：

Song, Baolin ^{[1
]}

Li, Ke ^{[1
]}

Zong, Yuan ^{[2
]}

Zhu, Jie ^{[1
]}

Zheng, Wenming ^{[2
]}

Shi, Jingang ^{[3
]}

Zhao, Li ^{[1
]}

机构：

[1] Southeast Univ, Sch Informat Sci & Engn, Key Lab Underwater Acoust Signal Proc, Minist Educ, Nanjing 210096, Peoples R China

[2] Southeast Univ, Key Lab Child Dev & Learning Sci, Minist Educ, Sch Biol Sci & Med Engn, Nanjing 210096, Peoples R China

[3] Univ Oulu, Ctr Machine Vis & Signal Anal, Oulu 90014, Finland

来源：

IEEE ACCESS | 2019年 / 7卷

基金：

中国国家自然科学基金;

关键词：

Micro-expression recognition; convolutional neural networks; apex frame location; spatiotemporal information; BINARY PATTERNS; RECOGNITION; FLOW;

D O I：

10.1109/ACCESS.2019.2960629

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Micro-expression recognition (MER) has attracted much attention with various practical applications, particularly in clinical diagnosis and interrogations. In this paper, we propose a three-stream convolutional neural network (TSCNN) to recognize MEs by learning ME-discriminative features in three key frames of ME videos. We design a dynamic-temporal stream, static-spatial stream, and local-spatial stream module for the TSCNN that respectively attempt to learn and integrate temporal, entire facial region, and facial local region cues in ME videos with the goal of recognizing MEs. In addition, to allow the TSCNN to recognize MEs without using the index values of apex frames, we design a reliable apex frame detection algorithm. Extensive experiments are conducted with five public ME databases: CASME II, SMIC-HS, SAMM, CAS(ME)(2), and CASME. Our proposed TSCNN is shown to achieve more promising recognition results when compared with many other methods.

引用

页码：184537 / 184551

页数：15

共 86 条

[31] Towards a dynamic expression recognition system under facial occlusion [J].

Huang, Xiaohua ;

Zhao, Guoying ;

Zheng, Wenming ;

Pietikainen, Matti .

PATTERN RECOGNITION LETTERS, 2012, 33 (16) :2181-2191

[32] Spatiotemporal Local Monogenic Binary Patterns for Facial Expression Recognition [J].

Huang, Xiaohua ;

Zhao, Guoying ;

Zheng, Wenming ;

Pietikainen, Matti .

IEEE SIGNAL PROCESSING LETTERS, 2012, 19 (05) :243-246

[33]

Iandola F.N., 2016, ARXIV160207360

[34] Facial Expression Analysis Based on High Dimensional Binary Features [J].

Kahou, Samira Ebrahimi ;

Froumenty, Pierre ;

Pal, Christopher .

COMPUTER VISION - ECCV 2014 WORKSHOPS, PT II, 2015, 8926 :135-147

[35]

Khor HQ, 2019, IEEE IMAGE PROC, P36, DOI [10.1109/icip.2019.8802965, 10.1109/ICIP.2019.8802965]

[36] Enriched Long-term Recurrent Convolutional Network for Facial Micro-Expression Recognition [J].

Khor, Huai-Qian ;

See, John ;

Phan, Raphael C. W. ;

Lin, Weiyao .

PROCEEDINGS 2018 13TH IEEE INTERNATIONAL CONFERENCE ON AUTOMATIC FACE & GESTURE RECOGNITION (FG 2018), 2018, :667-674

[37] Micro-Expression Recognition with Expression-State Constrained Spatio-Temporal Feature Representations [J].

Kim, Dae Hoe ;

Baddar, Wissam J. ;

Ro, Yong Man .

MM'16: PROCEEDINGS OF THE 2016 ACM MULTIMEDIA CONFERENCE, 2016, :382-386

[38] Micro-expression recognition based on 3D flow convolutional neural network [J].

Li, Jing ;

Wang, Yandan ;

See, John ;

Liu, Wenbin .

PATTERN ANALYSIS AND APPLICATIONS, 2019, 22 (04) :1331-1339

[39]

Li QY, 2018, INT C CONTROL DECISI, P265, DOI 10.1109/CoDIT.2018.8394868

[40]

Li XH, 2016, INT CONF SIGN PROCES, P1130, DOI 10.1109/ICSP.2016.7878004

← 1 2 3 4 5 6 7 8 9 →