Partial SRv6 Deployment and Routing Optimization: A Deep Reinforcement Learning Approach

Segment Routing over IPv6 (SRv6) is a promising efficient technology for traffic engineering (TE). As transitioning from a traditional distributed network to a full SRv6 network faces technical and economic challenges, partially deploying SRv6 has attracted much attention from academic communities. Many TE research attempts have been made on SRv6 deployment and routing optimization, among which Deep Reinforcement Learning (DRL) based algorithms have shown their advantages over traditional algorithms. However, with the incremental deployment of SRv6 nodes, training costs for DRL as well as solution space of routing optimization increase significantly, which obstructs the application of DRL-based algorithms in practice. To address this issue, we propose a DRL-based SRv6 deployment and routing optimization (SDRO) algorithm, with the TE objective of minimizing the maximum link utilization. In SDRO, the DRL agent is only trained once on a full SRv6 network and then used for different SRv6 deployment ratios. Hence, training costs can be obviously reduced. To reduce the solution space of routing optimization, the DRL agent performs routing pre-optimization on a portion of the traffic before routing is finally optimized by the Linear Programming method. By doing so, the execution time for routing optimization can be greatly reduced. Besides, for the issue of frequent link failures in the network, SDRO leverages the generalization of Graph Neural Networks to improve its robustness. Simulation results demonstrate that SDRO outperforms existing algorithms under different SRv6 deployment ratios and link failures, and completes routing optimization in a few seconds.