Multimodal emotion recognition based on peak frame selection from video

被引:24
|
作者
Zhalehpour, Sara [1 ]
Akhtar, Zahid [2 ]
Erdem, Cigdem Eroglu [3 ]
机构
[1] INRS EMT, Montreal, PQ, Canada
[2] Univ Udine, I-33100 Udine, Italy
[3] Bahcesehir Univ, Istanbul, Turkey
关键词
Affective computing; Facial expression recognition; Apex frame; Audio-visual emotion recognition; FUSION;
D O I
10.1007/s11760-015-0822-0
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
We present a fully automatic multimodal emotion recognition system based on three novel peak frame selection approaches using the video channel. Selection of peak frames (i.e., apex frames) is an important preprocessing step for facial expression recognition as they contain the most relevant information for classification. Two of the three proposed peak frame selection methods (i.e., MAXDIST and DEND-CLUSTER) do not employ any training or prior learning. The third method proposed for peak frame selection (i.e., EIFS) is based on measuring the "distance" of the expressive face from the subspace of neutral facial expression, which requires a prior learning step to model the subspace of neutral face shapes. The audio and video modalities are fused at the decision level. The subject-independent audio-visual emotion recognition system has shown promising results on two databases in two different languages (eNTERFACE and BAUM-1a).
引用
收藏
页码:827 / 834
页数:8
相关论文
共 50 条
  • [1] Multimodal emotion recognition based on peak frame selection from video
    Sara Zhalehpour
    Zahid Akhtar
    Cigdem Eroglu Erdem
    Signal, Image and Video Processing, 2016, 10 : 827 - 834
  • [2] Multimodal Emotion Recognition with Automatic Peak Frame Selection
    Zhalehpour, Sara
    Akhtar, Zahid
    Erdem, Cigdem Eroglu
    2014 IEEE INTERNATIONAL SYMPOSIUM ON INNOVATIONS IN INTELLIGENT SYSTEMS AND APPLICATIONS (INISTA 2014), 2014, : 116 - 121
  • [3] Improved TOPSIS method for peak frame selection in audio-video human emotion recognition
    Singh, Lovejit
    Singh, Sarbjeet
    Aggarwal, Naveen
    MULTIMEDIA TOOLS AND APPLICATIONS, 2019, 78 (05) : 6277 - 6308
  • [4] Improved TOPSIS method for peak frame selection in audio-video human emotion recognition
    Lovejit Singh
    Sarbjeet Singh
    Naveen Aggarwal
    Multimedia Tools and Applications, 2019, 78 : 6277 - 6308
  • [5] A Multimodal Emotion Recognition System from Video
    Thushara, S.
    Veni, S.
    PROCEEDINGS OF IEEE INTERNATIONAL CONFERENCE ON CIRCUIT, POWER AND COMPUTING TECHNOLOGIES (ICCPCT 2016), 2016,
  • [6] Multimodal emotion recognition based on feature selection and extreme learning machine in video clips
    Bei Pan
    Kaoru Hirota
    Zhiyang Jia
    Linhui Zhao
    Xiaoming Jin
    Yaping Dai
    Journal of Ambient Intelligence and Humanized Computing, 2023, 14 : 1903 - 1917
  • [7] Multimodal emotion recognition based on feature selection and extreme learning machine in video clips
    Pan, Bei
    Hirota, Kaoru
    Jia, Zhiyang
    Zhao, Linhui
    Jin, Xiaoming
    Dai, Yaping
    JOURNAL OF AMBIENT INTELLIGENCE AND HUMANIZED COMPUTING, 2021, 14 (3) : 1903 - 1917
  • [8] Video Emotion Recognition in the Wild Based on Fusion of Multimodal Features
    Chen, Shizhe
    Li, Xinrui
    Jin, Qin
    Zhang, Shilei
    Qin, Yong
    ICMI'16: PROCEEDINGS OF THE 18TH ACM INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, 2016, : 494 - 500
  • [9] Quality Based Frame Selection For Video Face Recognition
    Anantharajah, Kaneswaran
    Denman, Simon
    Sridharan, Sridha
    Fookes, Clinton
    Tjondronegoro, Dian
    6TH INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING AND COMMUNICATION SYSTEMS (ICSPCS'2012), 2012,
  • [10] Emotion recognition from multimodal physiological measurements based on an interpretable feature selection method
    Polo, Edoardo Maria
    Mollura, Maximiliano
    Lenatti, Marta
    Zanet, Marco
    Paglialonga, Alessia
    Barbieri, Riccardo
    2021 43RD ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE & BIOLOGY SOCIETY (EMBC), 2021, : 989 - 992