DeepDelivery: Leveraging Deep Reinforcement Learning for Adaptive IoT Service Delivery

To enable fast content delivery for delay-sensitive applications, large content providers build edge servers, Points of Presence (PoPs), and datacenters around the world. They are networked together as an integrated infrastructure via a private wide-area network (WAN), named content delivery network (CDN). To deliver quality services in the CDN, there are two critical decisions that should be properly made: 1) making assignments of PoP and datacenter for user requests, and 2) selecting routing paths from PoP to datacenter. However, with both the network variability and CDN environment complexity, it is challenging to achieve satisfying decisions. In this paper, we propose DeepDelivery, an adaptive deep reinforcement learning approach to intelligently make assignments and routing decisions in real time. Essentially, DeepDelivery adopts the Markov decision process (MDP) model to capture the dynamics of network variation, and the objective is to jointly maximize the infrastructure utilization of providers and minimize the total latency of end users. We conduct extensive trace-driven evaluations spanning various environment dynamics with both real-world and synthetic trace data. The result demonstrates that DeepDelivery can outperform the state-of-the-art scheme by 21.89% higher utilization and 11.27% lower end-to-end latency on average.