DDQP: A Double Deep Q-Learning Approach to Online Fault-Tolerant SFC Placement

Since Network Function Virtualization (NFV) decouples network functions (NFs) from the underlying dedicated hardware and realizes them in the form of software called Virtual Network Functions (VNFs), they are enabled to run in any resource-sufficient virtual machines. A service function chain (SFC) is composed of a sequential set of VNFs. As VNFs are vulnerable to various faults such as software failures, we consider how to deploy both active and standby SFC instances. Given the complexity and unpredictability of the network state, we propose a double deep Q-networks based online SFC placement scheme DDQP. Specifically, DDQP uses deep neural networks to deal with large continuous network state space. In the case of stateful VNFs, we offer constant generated state updates from active instances to standby instances to guarantee seamless redirection after failures. With the goal of balancing the waste of resources and ensuring service reliability, we introduce five progressive schemes of resource reservations to meet different customer needs. Our experimental results demonstrate that DDQP responds rapidly to arriving requests and reaches near-optimal performance. Specifically, DDQP outweighs the state-of-the-art method by 16.30% and 38.51% higher acceptance ratio under different schemes with 82x speedup on average. In order to enhance the integrity of the SFC state transition, we further proposed DDQP+, which extends DDQP by adding the delayed placement mechanism. Compared with DDQP, the design of the DDQP+ algorithm is more reasonable and comprehensive. The experiment results also show that DDQP+ achieved further improvement in multiple performance indicators.