Validation and Uncertainty in Model-Based Design Space Exploration - an Experience Report

Model-based systems engineering (MBSE) techniques can help manage the growing complexity in the design and development of cyber-physical systems, and can even allow for the optimization of a system under design in simulation. However, models are always an abstraction of the real-world systems they represent. This introduces uncertainty at the model level, which affects the validity of simulation results, and thus also the results of the optimization. This, together with variations in real-world system parameters, significantly complicates the validation of simulation and optimization results. In this experience report, we first use a descriptive process model to describe our efforts to validate the results of a model-based design space exploration (DSE) process given this uncertainty. After this, we discuss lessons learned and insights gained, and identify future challenges. We present a possible prescriptive process model for future validation efforts, which specifically takes into account uncertainty.