Autonomous Conflict Resolution in Urban Air Mobility: A Deep Multi-Agent Reinforcement Learning Approach

As urban air mobility (UAM) expands, electric vertical take-off and landing (eVTOL) vehicles are becoming increasingly prevalent, leading to new challenges in air traffic control. Specifically, managing eVTOL traffic at intersections and merging scenarios in low-altitude airspace becomes a safety-critical, real-time decision-making task due to the highly dynamic and stochastic nature of the environment. To address this challenge, we propose a deep multi-agent reinforcement learning framework specifically adapted to the complexities of eVTOL navigation in three-dimensional space. This framework employs asynchronous advantage actor critic (A3C), an actor-critic model, which is optimized to address the complexities of eVTOL navigation within three-dimensional space. We employ a centralized learning, decentralized execution strategy, utilizing a shared neural network across all agents in the environment. We validate the scalability and efficiency of our proposed framework in managing large-scale, high-density eVTOL traffic, ensuring safe separation and conflict resolution at intersections and merging points. Our model is extensively simulated in a suitable environment that accurately represents the physical dynamics of real-world environment. Preliminary results demonstrate the effectiveness of our framework in resolving nearly all conflicts in high-density traffic scenarios, promising a feasible solution for autonomous air traffic control in the rapidly developing UAM landscape.