Photogram Classification-Based Emotion Recognition

被引:5
作者
Miguel Lopez-Gil, Juan [1 ]
Garay-Vitoria, Nestor [1 ]
机构
[1] Univ Basque Country UPV EHU, Fac Informat, Donostiasan Sebastian 20018, Gipuzkoa Provin, Spain
来源
IEEE ACCESS | 2021年 / 9卷 / 09期
关键词
Emotion recognition; Face recognition; Databases; Videos; Feature extraction; Lighting; Affective computing; emotion recognition; classification algorithms; image classification; image sequence analysis; machine learning algorithms; FACIAL EXPRESSION RECOGNITION; SPEECH; DATABASE; SYSTEM; IMAGE;
D O I
10.1109/ACCESS.2021.3117253
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This paper presents a method for facial emotion recognition based on parameterized photograms and machine learning techniques. Videos of people displaying emotions are parameterized by a facial feature-based emotional category association process to determine whether a given photogram expresses emotions by comparing the facial action units displayed with findings in the literature about facial emotion. To test the proposed approach, two strategies are adopted. First, photograms displaying emotions are gathered, and then different machine learning classifiers are applied to check the goodness of the obtained set of categorized emotional photograms. Second, classifiers trained on the sets of emotional photograms were then used to emotionally classify all the videos in each database, using all the photograms with no preprocessing or photogram selection. The presented method was tested using the OpenFace parameterizer with emotional videos gathered from Multimedia Understanding Facial Expression (MUG) and Cohn-Kanade (CK+) databases. The outcomes achieved for emotional photogram classification on the sets of emotional photograms reached maximums of 99.80% and 99.63% in the MUG and CK+ databases, respectively. The videos were classified using different voting strategies regarding the outcome of each photogram in the video with all the photogram emotion recognition classifiers obtained results reflecting recognition rates of 70.71% and 66.36% for the videos in MUG and CK+ databases, and reached up to 72.55% and 88.37% when classifier combination strategies were used. The work carried out opens the door to follow-up work concerning data preprocessing and the use of different classifier combination methods in facial emotion recognition.
引用
收藏
页码:136974 / 136984
页数:11
相关论文
共 50 条
  • [41] Speech emotion recognition based on an improved brain emotion learning model
    Liu, Zhen-Tao
    Xie, Qiao
    Wu, Min
    Cao, Wei-Hua
    Mei, Ying
    Mao, Jun-Wei
    NEUROCOMPUTING, 2018, 309 : 145 - 156
  • [42] Context-Aware Emotion Recognition Based on Visual Relationship Detection
    Hoang, Manh-Hung
    Kim, Soo-Hyung
    Yang, Hyung-Jeong
    Lee, Guee-Sang
    IEEE ACCESS, 2021, 9 : 90465 - 90474
  • [43] Speech emotion recognition: Features and classification models
    Chen, Lijiang
    Mao, Xia
    Xue, Yuli
    Cheng, Lee Lung
    DIGITAL SIGNAL PROCESSING, 2012, 22 (06) : 1154 - 1160
  • [44] EEG-Based Emotion Classification Using Spiking Neural Networks
    Luo, Yuling
    Fu, Qiang
    Xie, Juntao
    Qin, Yunbai
    Wu, Guopei
    Liu, Junxiu
    Jiang, Frank
    Cao, Yi
    Ding, Xuemei
    IEEE ACCESS, 2020, 8 : 46007 - 46016
  • [45] Hierarchical Approach to Emotion Recognition and Classification in Texts
    Ghazi, Diman
    Inkpen, Diana
    Szpakowicz, Stan
    ADVANCES IN ARTIFICIAL INTELLIGENCE, PROCEEDINGS, 2010, 6085 : 40 - 50
  • [46] An integrated approach to emotion recognition and gender classification
    Rao, K. Prasada
    Rao, M. V. P. Chandra Sekhara
    Chowdary, N. Hemanth
    JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, 2019, 60 : 339 - 345
  • [47] EEG-based emotion recognition using hybrid CNN and LSTM classification
    Chakravarthi, Bhuvaneshwari
    Ng, Sin-Chun
    Ezilarasan, M. R.
    Leung, Man-Fai
    FRONTIERS IN COMPUTATIONAL NEUROSCIENCE, 2022, 16
  • [48] EEG Based Emotion Recognition: A Tutorial and Review
    Li, Xiang
    Zhang, Yazhou
    Tiwari, Prayag
    Song, Dawei
    Hu, Bin
    Yang, Meihong
    Zhao, Zhigang
    Kumar, Neeraj
    Marttinen, Pekka
    ACM COMPUTING SURVEYS, 2023, 55 (04)
  • [49] Emotion recognition based on multiple physiological signals
    Li, Qi
    Liu, Yunqing
    Yan, Fei
    Zhang, Qiong
    Liu, Cong
    BIOMEDICAL SIGNAL PROCESSING AND CONTROL, 2023, 85
  • [50] Emotion Recognition Based on Occluded Facial Expressions
    Ramirez Cornejo, Jadisha Yarif
    Pedrini, Helio
    IMAGE ANALYSIS AND PROCESSING,(ICIAP 2017), PT I, 2017, 10484 : 309 - 319