A pattern-based methodology for multimodal interaction design

This paper describes a design methodology for multimodal interactive systems. The method suggested is meant to serve as a foundation for the application of robust software engineering techniques in the field of multimodal systems. Starting from a short review of current design approaches we present a high level view of the design process for multimodal systems, highlighting design issues related to context of use factors. Our proposal is discussed in the context of a multimodal organizer which serves as our showcase application. The design of multimodal systems brings together a broad variety of analysis methods (task, context, data, user). The combination of modalities as well as the different interaction devices imply a high degree of freedom as far as design decisions are concerned. Therefore, a (simple) unification of existing approaches towards interface design like GOMS (task analysis) or Buergy's interaction constraint model for context analysis is not sufficient. We employ the design pattern approach as a means of guiding the analysis and design process. Design patterns are discussed as a general modeling tool as well as a possible approach towards designing multimodal systems.