A Holistic Auto-Scaling Algorithm for Multi-Service Applications Based on Balanced Queuing Network

Container-supported microservice technology is widely used in cloud applications. For elastic cloud, it's vital to maintain application response time within service-level agreements (SLA) by auto-scaling technology. For applications composed of multiple services (i.e. multi-service applications), due to complex topologies, there are many factors that reduce auto-scaling algorithm performance, such as correlations among services, untimely decision, oversupply, etc. To resolve this, we propose a holistic auto-scaling algorithm (HAB) based on balanced Jackson queuing network (JQN) to reduce SLA violations rapidly with less resource cost. With the holistic auto-scaling strategy, HAB scales all services quickly and accurately. Keeping the balanced state among services, HAB saves resource cost, reduces auto-scaling decision space and simplifies algorithm parameters. The experimental results demonstrate that HAB has an average decrease of 42.31% in SLA violation rate, an average decrease of 17.88% in resource cost and an average increase of 19.39% in stability, compared with other main methods.