Robustness enhancement of deep reinforcement learning-based traffic signal control model via structure compression

In recent years, deep reinforcement learning (DRL) has found extensive applications in the field of traffic signal control (TSC). However, many studies have demonstrated the vulnerabilities of DRL-based models exposed to abnormal data, which can lead to wrong knowledge representation. In this paper, an Integrated Structure and Robustness Enhancement framework is proposed, consisting of a Robust Distillation module and a Structure Compression module. Firstly, abnormal traffic flow data are utilized as training inputs generate a Poisoned Model, serving as a means to analyze potential data security threats. Secondly, the Robust Distillation module, which adopts a macro-level perspective, uses the latent knowledge in the hidden of the Poisoned Model as the distillation target. Small-gradient distillation is then performed by constructing a robust distillation loss function with temperature, through which wrong knowledge in the Poisoned Model is suppressed. Furthermore, the Structure Compression module, taking a micro-level perspective, conducts layer-wise quantification of redundant neurons in the robust student model and eliminates the abnormal parameters associated with abnormal data through iterative pruning of these redundant structures. Finally, streamlined and high-performance Enhancement Model is generated, containing only the minimum number neurons required. Experimental results demonstrate that the Integrated Structure and Robustness Enhancement framework effectively compresses the structure of the Poisoned Model, achieving minimal performance degradation and efficiently enhancing robustness compared to the Poisoned Model when exposed to abnormal data.