Verbal and non-verbal cues in the communication of emotions

Several studies have shown that humans extend their interpersonal behavioral patterns onto their interaction with computers. Based on this finding, research in human-computer interaction acknowledges the need to detect the users' expressions of emotion. However, we discovered that most of the current research is confined to emotion synthesis. In this investigation, we explored the role of verbal and non-verbal information in the communication of emotions. By training emotion-specific language and prosodic models on a corpus consisting of several thousands of sad, angry, or neutral speech segments from English movies, we showed that a classification system based on these models achieved an accuracy compareable to the accuracy of human listeners performing the same task.