Self-Prioritizing Multi-Agent Reinforcement Learning for Conflict Resolution in Air Traffic Control with Limited Instructions

Air traffic control (ATC) relies on a series of complex tasks, the most crucial aspect of which is to assure safe separation between aircraft. Due to the increase in air traffic, decision support systems and safe and robust automation of ATC tasks are of high value. Automated conflict resolution has been an active area of research for decades, and in more recent years, reinforcement learning has been suggested as a powerful alternative to traditional algorithms. Reinforcement learning using discrete action spaces often require large action spaces to cover all combinations of actions, which can make them difficult to train. On the other hand, models with continuous action spaces require much lower dimensionality but often learn to solve conflicts by using a large number of exceedingly small actions. This makes them more suitable for decentralized ATC, such as in unmanned or free-flight airspace. In this paper, we present a novel multi-agent reinforcement learning method with a continuous action space that significantly reduces the number of actions by means of a learning-based priority mechanism. We demonstrate how this can keep the number of actions to a minimum while successfully resolving conflicts with little overhead in the distance required for the aircraft to reach their exit points. As such, the proposed solution is well-suited for centralized ATC, where the number of directives that can be transmitted to aircraft is limited.