Why the Same Expression May Not Mean the Same When Shown on Different Faces or Seen by Different People

This somewhat sexist quote by Oscar Wilde points nonetheless to two important issues about faces. First, faces seem to tell us about who the other person is, and second, we may be misled by them. The present chapter has the goal to present some findings about the importance of faces in the context of emotion communication. Most research emotion considers the face as if it were a blank canvas with no meaning of its own. However, as we show, faces convey information relevant to the interpretation of emotion expressions and they have a meaning of their own which interacts with the facial expression shown. Because faces tell us about who the other person is, we may also not use the same cues when interpreting the facial expressions of different people. That is, when it comes to interpreting emotional facial expressions it really matters who shows what to whom and in which context (Hess, Beaupre, & Cheung, 2002). This in turn has relevance for the use of emotion expressions in human-machine interfaces. Agents are made to express emotions so as to facilitate communication by making it more like human-human communication (Koda & Maes, 1996: Pelachaud & Bilvi, 2003). However, this very naturalness may mean that an agent may fail to convey the intended message because of the way it looks and the biases in perception and interpretation that this may entrain. It is therefore important to not only focus creating believable expressions but also to keep in mind that the face which these expressions appear is not an empty canvas. In what follows we present two examples of how faces interact with emotion expressions and how information transmitted by faces may change the way facial cues are interpreted.