Emotion Recognition Research based on Integration of Facial Expression and Voice

Emotion recognition is becoming more and more important with the growing interests on intelligent and natural human-machine interaction. This paper presents a research on emotion recognition based on the combination of facial expression and human voices. The geometry shape of facial organs, represented by cubic spline coefficients, as well as the facial texture, represented by Histogram of Gradients (HOG), are combined together to describe the emotion shown on human face. A classifier for facial expression-based emotion recognition is trained using Support Vector Machine (SVM) with the image data provided by CHEAVD 2.0. Meanwhile, another voice-based emotion classifier is trained also using SVM with acoustic features extracted from the voices accompanying the video. The recognition results are then combined at decision-level using Bayesian rule. This approach can achieve an accuracy of 38.41% with the test data of CHEAVD 2.0. More details of the testing result are shown in the paper. In the end, pros and constraints of our approach are discussed.