Comprehensive Study of Features for Subject-independent Emotion Recognition

In this paper, we conduct a comprehensive study to identify the most discriminative features that address the interpersonal variability to perform efficient human emotion recognition task. We consider three commonly used feature extraction techniques, namely, the Local Binary Patterns (LBP), the Scale-Invariant Feature Transform (SIFT) and the curvelet transforms to extract features from the images on the JAFFE data set. A subset of these features is then selected using the Double Input Symmetrical Relevance (DISR), the Conditional Mutual Information Maximization (CMIM) and the minimum Redundancy maximum Relevance (mRMR) methods. The original feature sets and the subsets are then used to train a PBL-McRBFN classifier. We conduct a subject independent study with 10 cross validations on the JAFFE data set. The average performance of the PBL-McRBFN classifier with the different feature sets and subsets are compared. In general, feature selection methods used along with the feature extraction techniques help to perform emotion recognition more efficiently. It is also observed that the subset of features selected using the mRMR on the features extracted from the SIFT technique (SIFT+mRMR+PBL-McRBFN) is the most discriminative feature subset. A statistical paired t-test also ascertains this observation. We also compare the performance of the SIFT+mRMR+PBL-McRBFN with the other results in the literature for this problem. Performance comparison shows that the SIFT+mRMR+PBL-McRBFN outperforms other state-of-the-art methods in the literature for this problem.