A Component Based Approach for Classifying the Seven Universal Facial Expressions of Emotion

In a previous publication we showed a component approach performing better than a holistic approach for classifying 6 discrete facial expressions of emotion (happy, sad, surprised, fearful, disgusted and angry) plus the neutral face. In this paper, we present the impact on the two approaches if a seventh facial expression is included. This seventh expression is known as the contemptuous expression (an expression that has rarely been explored in the machine learning literature). Together, the 7 facial expressions are known in the psychology literature as Ekman's Universal Facial Expressions of Emotion. To replicate our previous experiments, we use the same number of image preprocessing techniques (grayscale, Local Binary Patterns, Sobel and Canny edge detection, and manually inserted feature points which we refer to as components) that are organised into four approaches: holistic, holistic action, component and component action (where an action approach is produced by a difference between a neutral and an expressive face). The facial expressions are classified using a standard multiclass Support Vector Machine (SVM) and a pairwise adaptive multiclass SVM (pa-SVM) - which uses pairwise adaptive model parameters. The significance of this study is to provide a thorough understanding of the choices that impact a classifier's performance and the performance dynamics between expression pairs when the contemptuous expression is now considered. In particular, we extend the issues and justify the use of the contemptuous expression by noting its universal presence in human expression and its asymmetric nature, which is not shared with other expressions. Using our face model, which uses feature points around the eyes, brows and mouth, we obtained a best correct classification rate of 98.57% with the contemptuous expression included.