Modeling multimodal human-computer interaction: Semiotics, proxemics and kinesics

In multimedia computing environments, user interfaces need to deal with input and output from diverse data modalities such as text, sound/speech, images and pointing devices. In theater and drama, information is presented to the audience via oral (and, in some cases, written) text, gestures and stage settings. By viewing theater and drama as metaphors for human-computer communication, some of the theoretical approaches used in the analysis of the communicational aspects of theater and drama can also be advantageously applied to the modeling and analysis of human-computer interaction.