A Self-attention Agent of Reinforcement Learning in Continuous Integration Testing

Test case prioritization based on reinforcement learning has been seen as a promising way to achieve continuous integration testing. Agent and reward function are two crucial components of reinforcement learning. During the process of reinforcement learning in continuous integration test case prioritization, the agent decides on the execution order of test cases (actions) for the next integration testing (environment) based on the corresponding test case features (states), aiming to detect errors early by maximizing the reward. Furthermore, having more test case features allows the agent to perceive the environment better, but it also increases computation consumption and brings convergence problems to learning. In this paper, we first propose a multi-feature environment perception for continuous integration test case prioritization. It introduces multiple features based on test case history execution information to solve the agent's limitation in obtaining environmental information. Additionally, we propose a self-attention agent network structure, which captures relationships between multiple features to prevent the convergence problem of reinforcement learning. An extensive experimental and analytical study was conducted with 15 existing reward functions on 14 industrial data sets. The results show that (1) the proposed multiple features can help the agent to perceive environmental information more comprehensively, and (2) the proposed self-attention agent can process environmental information better to achieve more effective test case prioritization in continuous integration testing.