Cdascaler: a cost-effective dynamic autoscaling approach for containerized microservices

Microservices are containerized, loosely coupled, interactive smaller units of the application that can be deployed, reused, and maintained independently. In a microservices-based application, allocating the right computing resources for each containerized microservice is important to meet the specific performance requirements while minimizing the infrastructure cost. Microservices-based applications are easy to scale automatically based on incoming workload and resource demand automatically. However, it is challenging to identify the right amount of resources for containers hosting microservices and then allocate them dynamically during the auto-scaling. Existing auto-scaling solutions for microservices focus on identifying the appropriate time and number of containers to be added/removed dynamically for an application. However, they do not address the issue of selecting the right amount of resources, such as CPU cores, for individual containers during each scaling event. This paper presents a novel approach to dynamically allocate the CPU resources to the containerized microservice during the autoscaling events. Our proposed approach is based on the machine learning method, which can identify the right amount of CPU resources for each container, dynamically spawning for the microservices over time to satisfy the application's response time requirements. The proposed solution is evaluated using a benchmark microservices-based application based on real-world workloads on the Kubernetes cluster. The experimental results show that the proposed solution outperforms by yielding a 40% to 60% reduction in violating the response time requirements with 0.5x to 1.5x less cost compared to the state-of-art baseline methods.