Deep Reinforcement Learning Approach to Air Traffic Optimization Using the MuZero Algorithm

Urban Air Mobility (UAM) and Unmanned Aircraft Systems (UAS) are anticipated to result in a significant growth in air traffic that will require novel Air Traffic Management (ATM) solutions to problems such as path planning with separation constraints. In this paper, we apply MuZero, a newly-introduced deep reinforcement learning algorithm by DeepMind [1] to path planning problems in dynamic air traffic environments. MuZero has demonstrated exceptional progress in Artificial Intelligence (AI) game playing. To formulate the path planning problem, we consider a sequential trajectory allocation approach that would act on a "first-come-first-serve" basis for both online planning and moving time horizon problems. Initial results show that agents can learn to mitigate collisions when trained with the obstacle avoidance framework based on the MuZero algorithm without requiring any knowledge about the domain and game rules.