Reinforcement learning for multi-agent formation navigation with scalability

This paper addresses the multi-agent formation obstacle avoidance (MAFOA) problem using multi-agent deep reinforcement learning (MADRL). MAFOA control aims to achieve and maintain a desired formation while avoiding collisions among agents or with obstacles. It is a research hotspot in multi-agent cooperation due to its wide applications and challenges. However, current MADRL methods face two major difficulties in solving this problem: 1) the high complexity and uncertainty of the environment when there are many agents; 2) the lack of scalability when the number of agents varies. To overcome these difficulties, we propose: 1) A local multi-agent deep deterministic policy gradient algorithm that allows each agent to learn from its local neighbors' strategies during training and act independently during execution; 2) A reinforcement learning framework based on local information that uses partial observation as input and adapts to different numbers of agents; 3) A hybrid control method that switches between reinforcement learning and PID control to ensure formation stability. We evaluate our method on the multiagent particle environment environment and compare it with other algorithms to demonstrate its feasibility and superiority for solving the MAFOA problem.