A Comparison of Facial Features and Fusion Methods for Emotion Recognition

Emotion recognition is an important part of human behavior analysis. It finds many applications including human-computer interaction, driver safety, health care, stress detection, psychological analysis, forensics, law enforcement and customer care. The focus of this paper is to use a pattern recognition framework based on facial expression features and two classifiers (linear discriminant analysis and k-nearest neighbor) for emotion recognition. The extended Cohn-Kanade database is used to classify 5 emotions, namely, 'neutral, angry, disgust, happy, and surprise'. The Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), the Walsh-Hadamard Transform (FWHT) and a new 7-dimensional feature based on condensing the Facial Action Coding System (FACS) are compared. Ensemble systems using decision level, score fusion and Borda count are also studied. Fusion of the four features leads to slightly more than a 90% accuracy.