AAM-based emotion recognition using variance of facial feature points on mobile video stream

Understanding and classifying human emotion can play an important roles in the interaction between human and machine communication system. The most expressive way to display the human emotion is through facial expression analysis. In this paper, we propose a novel extraction and recognition method of facial expression and emotion from mobile camera through the proposed classifier. Especially, we formulate a new classification model of facial emotions using a variance of the estimated landmark points. We utilize the variance values of 65 feature point locations to recognize a facial emotion by comparing with weighted fuzzy k-NN classification between on the current frame and on the previous frame. Finally, five types of facial emotions are recognized and classified as a facial expression in neutral, happy, angry, surprise or sad. To evaluate the performance of the proposed algorithm, we assess the ratio of success with iPhone/iPad camera views. The experimental results show that the proposed method effectively performed well in the recognition of facial emotion, and the obtained result indicates the good performance and enough to applicable to mobile environments.