Evaluation of spoken language understanding and dialogue systems

A spoken language understanding and dialogue system in the domain of appointment schedule is presented. The system is capable of understanding complex times, e.g. it correctly combines discontinuous constituents and resolves ambiguities. A distributed representation of surface structure models and an incremental semantic analysis is used to manage the complexity. An elaborate evaluation of the system based on measurements of accuracy was carried out. Our approach combines pattern recognition with linguistic aspects forming a system of measurement consisting of word accuracy, constituent accuracy, and concept accuracy.