Design of explanation user interfaces for interactive machine learning using the example of a knowledge graph-based approach to explainable process analysis

The practical use of AI technologies with user interactions (e.g. in the form of self-service applications in consulting) require users to be able to understand and comprehend the results generated. A knowledge graph-based approach to process analyses with interactive machine learning methods identifies weaknesses and suitable improvement measures in business processes. In order to present the analysis results in a user-understandable way, e.g. for consulting clients, and to enable verification and corrections by expert users, an explainable and user-friendly interface is required. While many explainable AI researchers deal with computational aspects of generating explanations, there is less research on the design of eXplanation User Interfaces (XUI). In this paper, a systematic literature review identifies 41 XUIs for interactive machine learning, deriving design components and summarizing them in a design catalog, which forms the basis for specifying requirements on these interfaces. For evaluation purposes, requirements objectives regarding an XUI for the knowledge graph-based approach of process analysis were defined and specified with the help of selected design components from the design catalog. The requirements specifications were afterwards implemented and demonstrated using an example process. An evaluation with process analysts and consultants shows that it depends not on a high number of implemented design components, but rather on a careful selection of different forms of explanation (e.g. visual, textual) for both local and global explanation content in order to present analysis results in a comprehensible and understandable way. XUI with interaction functions for verifying and correcting analysis results increase the willingness to use AI systems. This can help to improve the acceptance of AI technologies in day-to-day consulting for both consultants and their clients.