From Explainable AI to Explainable Simulation: Using Machine Learning and XAI to understand System Robustness

Evaluating robustness is an important goal in simulation-based analysis. Robustness is achieved when the controllable factors of a system are adjusted in such a way that any possible variance in uncontrollable factors (noise) has minimal impact on the variance of the desired output. The optimization of system robustness using simulation is a dedicated and well-established research direction. However, once a simulation model is available, there is a lot of potential to learn more about the inherent relationships in the system, especially regarding its robustness. Data farming offers the possibility to explore large design spaces using smart experiment design, high performance computing, automated analysis, and interactive visualization. Sophisticated machine learning methods excel at recognizing and modelling the relation between large amounts of simulation input and output data. However, investigating and analyzing this modelled relationship can be very difficult, since most modern machine learning methods like neural networks or random forests are opaque black boxes. Explainable Artificial Intelligence (XAI) can help to peak into this black box, helping us to explore and learn about relations between simulation input and output. In this paper, we introduce a concept for using Data Farming, machine learning and XAI to investigate and understand system robustness of a given simulation model.