Traffic Signal Control Method Based on Modified Proximal Policy Optimization

For the traffic congestion problem at intersections, this paper proposed a modified proximal policy optimization algorithm, which can adaptively regulate traffic signals to alleviate intersection congestion. The algorithm made further theoretical corrections to the unbiasedness of the advantage function in it through the importance sampling method. The algorithm was also modified to better adapt to traffic signal control and thus increase the sensitivity of the algorithm to control actions. In order to fully extract road state information and reduce the complexity of the state space, the experiment used images as state inputs, which are real-time snapshots of road conditions at intersections. To verify the performance of the proposed algorithm, it was compared with other deep reinforcement learning algorithms in the traffic simulation software SUMO. The results show that the cumulative reward of the model constructed by the proposed algorithm is increased by 68.3% and the average waiting time is reduced by 65.3% compared to the baseline DQN model. Experimental results demonstrate that the algorithm can effectively regulate traffic congestion.