Deep-Reinforcement-Learning-Based Design Space Exploration for Time-Sensitive Networking

Time-Sensitive Networking (TSN) has emerged as a favorable option for real-time communication in Cyber-Physical Systems (CPSs), such as intelligent vehicles and industrial control systems, due to its capability of providing bounded end-to-end latencies. However, designing CPSs on a TSN network requires additional consideration of data flow schedulability and becomes an NP-hard combinatorial optimization problem. Therefore, a formal and efficient approach is desired to explore the design space with different combinations of data flow periods. Accordingly, we propose a novel design flow that preemptively generates a set of schedulable period lists, guaranteeing that all data flow deadlines can be met, to preclude the schedulability concern. We further employ Deep Reinforcement Learning (DRL) to optimize the searching process of these period lists. The experimental result demonstrates remarkable success where 97.02% solutions are found with 4.85x speed higher than a Satisfiability-Modulo-Theories-based method. The result of large-scale scenarios also reveals that our approach outperforms any other comparative method by at least 3.93x more schedulable period lists collected.