Multi-Objective Black-Box Test Case Selection for System Testing

Testing is a fundamental task to ensure software quality. Regression testing aims to ensure that changes to software do not introduce new failures. As resources are often limited and testing comprises a vast amount of test cases, different regression strategies have been proposed to reduce testing effort by selecting or prioritizing important test cases, e.g., code coverage (to ensure a sufficient testing depth). However, in system testing, source code is often not available creating a black-box system. In this paper, we introduce an automated, multi-objective test case selection technique in black-box systems using genetic algorithms. We define seven different objectives, based on meta-data, allowing a flexible test case selection for a variety of systems. For evaluation, we apply our technique on two different subject systems assessing the feasibility and suitability of our test case selection approach. Results indicate that our approach is applicable based on different data available and is able to outperform random test case selection and retest-all.