Facial expression is one of the most powerful channels of nonverbal communication which contains plenty of affective information. Recognition of facial expression and sending them back to the teacher is potentially helpful in E-learning. In this paper, we differentiate between person-relevant and person-irrelevant situations. Our goal is to extract powerful features used for facial expression recognition system in real-time and person-irrelevant situation. Previous work suggests that both facial shape features and appearance features could be used to recognize facial expressions. The first type is shape features calculated from positions on a face. The second type is a set of multi-scale and multi-orientation Gabor wavelet coefficients. The classifier is based on Support Vector Machines (SVM) and our expriments cover both person-relevant and person-irrelevant situations. The result shows that in person-irrelevant situation, using facial shape features outperforms using Gabor wavelet and it is faster. Furthermore, the radial basis function of SVM is more suitable for person-associated situation and the linear function describes person-irrelevant problems better. (C) 2012 Published by Elsevier B.V. Selection and peer review under responsibility of Information Engineering Research Institute