Deep reinforcement learning in real-time strategy games: a systematic literature review

Reinforcement learning is a field of Machine Learning in which agents learn from interacting with the environment. These agents can deal with more complex problems when their decision-making process is combined with deep learning. While deep reinforcement learning can be used in many real-world applications, games often provide a good source of simulation environments for testing such algorithms. Among all game categories, real-time strategy games usually pose a difficult challenge since they have large state and action spaces, partial observation maps, sparse reward, and Multi-Agent problems, where the events occur continuously simultaneously. Thus, this paper provides a systematic literature review of deep reinforcement learning related to real-time strategy games. The main goals of this review are presented as follows: (a) identify the games used in recent works; (b) summarize the architectures and techniques used; (c) identify the simulation environments adopted and (d) understand whether the works focus on micromanagement or macromanagement tasks when dealing with real-time strategy games. The results show that some architectures have achieved better performance overall when handling both micro and macromanagement tasks, and that techniques for reducing the training time and the state space may improve the agents learning. This paper may help to guide future research on developing strategies to build agents for complex scenarios such as those faced in real-time strategy games.Graphical abstractVisual summary of the Systematic Literature Review methodology and results. It presents the objective of the review, the research questions, the protocol parameters and criteria, and the results