Multi-agent Reinforcement Learning in a Large Scale Environment via Supervisory Network and Curriculum Learning

Multi-agent reinforcement learning is essential for learning optimal policy for collaboration and competition environments. However, as the action space of the agent increases, the number of state-action pairs which have to be explored increases exponentially. As a result, increasing search space causes difficulty to converge the learning. To solve this problem, we propose a supervisory network. To achieve the global goal, the supervisory network creates a sub-goal and assigns the goals to the agents so that the agents can effectively learn the optimal policy with a small action space. In addition, we adapt the curriculum learning method to learn a large-scale environment. As a consequence, the agent can explore the environment in which the complexity increases gradually. Although a baseline network was learned in the same environment to compare with our model, the baseline fails to learn an optimal policy while our model successes to learn in the large-scale environment.