Applying simulation and design of experiments to the embedded software testing process

This paper presents some original solutions with regard to the deployment of the U.S. Department of Defense Simulation, Test and Evaluation Process (DoD STEP), using an automated target tracking radar system as a case study. Besides the integration of modelling and simulation, to form a model-based approach to the software testing process, the number of experiments, i.e. test cases, have been dramatically reduced by applying an optimized design-of-experiment plan and an orthogonal array-based robust testing methodology. Also, computer-based simulation at various abstraction levels of the system/software under test can serve as a test oracle. Simulation-based (stochastic) experiments, combined with optimized design-of-experiment plans, in the case study have shown a minimum productivity increase of 100 times in comparison to current practice without DoD STEP deployment. Copyright (C) 2004 John Wiley Sons, Ltd.