Test Suite Reduction by Combinatorial-based Coverage of Event Sequences

Combinatorial-based criteria are useful in several studies for test suite generation, prioritization, and minimization. In this paper, we extend previous work by using combinatorial-based criteria for test suite reduction. We use criteria that are based on combinatorial coverage of events and consider the order in which events occur. A simple combinatorial-based criterion covers t-way events and does not differentiate between the order of events. The event pair (e(1),e(2)) is counted the same as if it occurs in the order (e(2),e(1)). We also use two criteria that consider the order in which events occur since different orderings of events may be valuable during testing. First, the consecutive sequence-based criterion counts all event sequences in different orders, but they must occur adjacent to each other. The sequence-based criterion counts pairs in all orders without the requirement that events must be adjacent. We evaluate the new criteria on three GUI applications. We use 2way inter-window coverage in our studies. All of the 2way combinatorial-based criteria are effective in reducing the test suites and maintaining close to 100% fault finding effectiveness. Our future work examines larger test suites, higher strength coverage, techniques to partition event data, and further empirical studies.