DRL-QOR: Deep Reinforcement Learning-Based QoS/QoE-Aware Adaptive Online Orchestration in NFV-Enabled Networks

Faced with fluctuating network traffic and unknown underlying network traffic dynamics, developing an effective orchestration model with low network cost is still a critical issue in Network Functions Virtualization (NFV)-enabled networks. Thus we propose a Deep Reinforcement Learning based Quality of Service (QoS)/Quality of Experience (QoE)-Aware Adaptive Online Orchestration (DRL-QOR) approach to adapt to the real- time network variations. We formulate the stochastic resource optimization as a Parameterized Action Markov Decision Process (PAMDP), with QoE and specific QoS requirements as key factors in formulating the reward function, aiming to maximize QoE while satisfying QoS constraints. Then we propose DRL-QOR to solve the Non-deterministic Polynomial hard (NP-hard) problem with consideration of improving the long-term profits, where deep neural network combinatorial optimization theory is extended under the constraints of the binary integer programming model. Extensive experimental results in real USANET topology demonstrate that our proposed DRL-QOR converges fast during the training process. Compared with other benchmarks that only consider the current system performance, it shows good performance in QoE provisioning and QoS requirements maintenance for orchestrating SFCs.