Traffic signal optimization control method based on adaptive weighted averaged double deep Q network

As a critical node and major bottleneck of the urban traffic networks, the control of traffic signals at road intersections has an essential impact on road traffic flow and congestion. Deep reinforcement learning algorithms have shown excellent control effects on traffic signal timing optimization. Still, the diversity of actual road control scenarios and real-time control requirements have put forward higher requirements on the adaptiveness of the algorithms. This paper proposes an Adaptive Weighted Averaged Double Deep Q Network (AWA-DDQN) based traffic signal optimal control method. Firstly, the formula is used to calculate the double estimator weight for updating the network model. Then, the mean value of the action evaluation is calculated by the network history parameters as the target value. Based on this, a certain number of adjacent action evaluation values are used to generate hyperparameters for weight calculation through the fully connected layer, and the number of action values for mean calculation is gradually reduced to enhance the stability of model training. Finally, simulation experiments were conducted using the traffic simulation software Vissim. The results show that the AWA-DDQN-based signal control method effectively reduces the average delay time, the average queue length and the average number of stops of vehicles compared with existing methods, and significantly improves traffic flow efficiency at intersections.