Decentralized multi-region perimeter control in complex urban environments using reinforcement and imitation learning

Multi-region Perimeter Control is a promising traffic control approach, effectively regulating vehicle flow between congested regions to improve travel efficiency and prevent network oversaturation. Recent advancements in AI and deep reinforcement learning (DRL) have led researchers to explore the application of DRL for perimeter control applications. This study introduces two complementary methods for multi-region perimeter control. The first is a centralized and model-free DRL controller that operates without requiring prior knowledge of a macroscopic fundamental diagram (MFD). The second one involves decentralization at two different levels, i.e., region- and boundary-level, achieved by using multi-agent imitation learning aiming to decentralize the policy of the centralized controller. Both approaches are augmented by a queue balancing framework and operate in a hierarchical manner. The queue balancing framework is responsible for the translation of ordered inflows to optimal green time durations based on relative queues to account for the heterogeneity found in real-world networks. The proposed controllers, trained directly in a realistic large-scale microsimulation network, are evaluated and compared against two widely used perimeter control methods found in literature, i. e., a multivariable Proportional-Integral (PI) feedback regulator and a model predictive control approach. Results show that both proposed methods outperform the baselines in terms of efficiency. Furthermore, generalization tests under various synthetic demand scenarios, including scenarios affected by signal noise, demonstrate the robustness of the learned policies. These findings highlight the potential of imitation learning to effectively decentralize DRL-based perimeter control while maintaining high performance levels.