Deep Reinforcement Learning Based Dynamic Routing Optimization for Delay-Sensitive Applications

With the rapid development of the Internet and the approaching of the next-generation networking, the number and variety of delay-sensitive applications have increased dramatically. Nowadays, how to properly route delay-sensitive packets in complex network environment and meet the stringent quality-of-service (QoS) requirements of delay-sensitive applications remains a great challenge. Towards this end, this paper proposes a deep reinforcement learning (DRL)-based routing algorithm for delay-sensitive applications featuring the proximal policy optimization (PPO) method and the front-convergent actor-critic network (FCACN) technique. To meet the high demand of delay-sensitive applications, we consider the packet survival time (ST) to help our algorithm perform better and make up for the shortage of the time-to-live (TTL) mechanism in IP network. We conduct extensive experiments to prove the efficiency and reliability of the proposed algorithm. Experimental results show that the proposed algorithm outperforms two traditional routing protocols and two state-of-the-art DRL-based routing algorithms in terms of minimizing delay and packet loss rate.