SimEvo: Testing Evolving Multi-Process Software Systems

Regression testing is used to perform re-validation of evolving software. However, most existing techniques for regression testing focus exclusively on single-process applications, but to date, no work has considered regression testing for software involving multiple processes or event handlers (e.g., software signals) at the system-level. The unique characteristics of concurrency control mechanism employed at the system-level can affect the static and dynamic analysis techniques on which existing regression testing approaches rely. Therefore, applying these approaches can result in inadequately tested software during maintenance, and ultimately impair software quality. In this paper, we propose SimEvo, the first regression testing techniques for multi-process applications. SimEvo employs novel impact analysis techniques to identify system-level concurrent events that are affected by the changes. It then reuses existing test cases, as well as generating new test cases, focused on the set of impacted events, to effectively and efficiently explore the newly updated concurrent behaviors. Our empirical study on a set of real-world Linux applications shows that SimEvo is more cost-effective in achieving high inter-process coverage and revealing real world system-level concurrency faults than other approaches.