Online Deadline-aware Bulk Transfer over Inter-Datacenter WANs

Many large-scale compute-intensive and mission critical online service applications are being deployed on geo-distributed datacenters, which require transfers of bulk business data over Wide Area Networks (WANs). The bulk transfers are often associated with different requirements on deadlines, either a complete transfer before a hard deadline or a best-effort delivery within a soft deadline. In this paper, we study the online bulk transfer problem over inter-datacenter WANs, while taking into consideration the requests with a mixture of hard and soft deadlines. We use Linear Programming (LP) to mathematically formulate the problem with the objective of maximizing a system utility represented by the service provider's revenue, taking into account the revenue earned from deadline-met transfers and the penalty paid for deadline-missed ones. We propose an online framework to efficiently manage mixed bulk transfers and design a competitive algorithm that applies the primal-dual method to make routing and resource allocation based on the LP. We perform theoretical analysis to prove that the proposed approach can achieve a competitive ratio of (e - 1)/e with little link capacity augmentation. In addition, we conduct comprehensive simulations to evaluate the performance of our method. Simulation results show that our method irrespective of the revenue model, can accept at least 25% more transfer requests and improve the network utilization by at least 35%, compared to prior solutions.