Freeway Ramp-Metering Control based on Reinforcement Learning

Random occurrences of traffic congestion on freeways lead to system degradation over time. If no smart control measures are applied, this degradation can lead to accumulated congestion which can severely affect other parts of the traffic network. Consequently, the need for an optimal and reliable traffic control has become more critical. The aim of this research is to control the amount of vehicles entering the mainstream freeway from the ramp merging area, i.e., balance the demand and the capacity of the freeway. This keeps the freeway density below the critical density. Consequently, this leads to maximum utilization of the freeway without entering in congestion while maintaining the optimal freeway operation. The Reinforcement learning based density control agent (RLCA) is designed based on Markovion modeling with an associated Q-learning algorithm in order to address the stochastic nature of the traffic situation. Extensive analysis is conducted in order to assess the proposed definition of the (state, action) pairs, as well as the reward function. We experiment with two case studies with two different network structures and demands. The first case study, which is the benchmark network used in literature, is the network with dense demand. Whereas the other one is the network with light demand. RLCA shows a superior response with respect to a predetermined reference points especially in terms of freeway density, flow rate, and total travel time.