An interactive greeting system using convolutional neural networks for emotion recognition

被引：10

作者：

Lu, Ching-Ta ^{[1
,2
,3
]}

Su, Chung-Wei ^{[1
]}

Jiang, Hui-Ling ^{[1
,4
]}

Lu, Yen-Yu ^{[1
]}

机构：

[1] Asia Univ, Dept Informat Commun, Taichung 41354, Taiwan

[2] Asia Univ, Dept Audiol & Speech Language Pathol, Taichung 41354, Taiwan

[3] China Med Univ, China Med Univ Hosp, Dept Med Res, Taichung 40402, Taiwan

[4] Natl Yang Ming Chiao Tung Univ, Dept Commun & Technol, Hsinchu 302, Taiwan

来源：

ENTERTAINMENT COMPUTING | 2022年 / 40卷

关键词：

Deep-learning; Convolutional neural network; Emotion recognition; Interactive greeting; Human-computer interaction;

D O I：

10.1016/j.entcom.2021.100452

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

The convolutional neural network (CNN) has been progressively applied in face recognition and various applications. This paper presents an interactive greeting system that combines face identification, emotion recognition, and funny 3D animation. The proposed system recognizes a user by a face-CNN and interacts with the user according to the recognized mood. The animated 3D girl plays a motion-dependent video according to the recognized user's emotion. First, we employ a Viola-Jones algorithm to segment the areas around the face, eyes, and mouth in a captured image. The face-CNN recognizes a known person. An eye-CNN and a mouth-CNN recognize the emotion. Finally, a 3D animation is played according to the identified emotion. Experimental results show that the proposed interactive greeting system recognizes target persons well, identifies the emotion adequately, and provides users to enjoy funny interaction.

引用

页数：11

共 29 条

[1] APA: Adaptive Pose Alignment for Pose-Invariant Face Recognition
An, Zhanfu
Deng, Weihong
Hu, Jiani
Zhong, Yaoyao
Zhao, Yuying
[J]. IEEE ACCESS, 2019, 7 : 14653 - 14670
[2] Modeling and adjusting in-game difficulty based on facial expression analysis
Blom, Paris Mavromoustakos
Bakkes, Sander
Spronck, Pieter
[J]. ENTERTAINMENT COMPUTING, 2019, 31
[3] Affective Computing and Sentiment Analysis
Cambria, Erik
[J]. IEEE INTELLIGENT SYSTEMS, 2016, 31 (02) : 102 - 107
[4] Fuzzy commonsense reasoning for multimodal sentiment analysis
Chaturvedi, Iti
Satapathy, Ranjan
Cavallari, Sandro
Cambria, Erik
[J]. PATTERN RECOGNITION LETTERS, 2019, 125 : 264 - 270
[5] Change Detection in Multisource VHR Images via Deep Siamese Convolutional Multiple-Layers Recurrent Neural Network
Chen, Hongruixuan
Wu, Chen
Du, Bo
Zhang, Liangpei
Wang, Le
[J]. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2020, 58 (04): : 2848 - 2864
[6] Multimedia Deep Learning
Chen, Shu-Ching
[J]. IEEE MULTIMEDIA, 2019, 26 (01) : 5 - 7
[7] Trunk-Branch Ensemble Convolutional Neural Networks for Video-Based Face Recognition
Ding, Changxing
Tao, Dacheng
[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2018, 40 (04) : 1002 - 1014
[8] Multimodal shared features learning for emotion recognition by enhanced sparse local discriminative canonical correlation analysis
Fu, Jiamin
Mao, Qirong
Tu, Juanjuan
Zhan, Yongzhao
[J]. MULTIMEDIA SYSTEMS, 2019, 25 (05) : 451 - 461
[9] Ghofrani A, 2019, 2019 IEEE 5TH CONFERENCE ON KNOWLEDGE BASED ENGINEERING AND INNOVATION (KBEI 2019), P817, DOI [10.1109/KBEI.2019.8734924, 10.1109/kbei.2019.8734924]
[10] Affective content analysis of music emotion through EEG
Hsu, Jia-Lien
Zhen, Yan-Lin
Lin, Tzu-Chieh
Chiu, Yi-Shiuan
[J]. MULTIMEDIA SYSTEMS, 2018, 24 (02) : 195 - 210

← 1 2 3 →