Learning Predictive Auto scaling Policies for Cloud-hosted Microservices Using Trace-driven Modeling

Autoscaling methods are important to ensure response time guarantees for cloud-hosted microservices. Most of the existing state-of-the-art autoscaling methods use rule-based reactive policies with static thresholds defined either on monitored resource consumption metrics such as CPU and memory utilization or application-level metrics such as the response time. However, it is challenging to determine the most appropriate threshold values to minimize resource consumption and performance violations. Whereas, predictive autoscaling methods can help to address these challenges. These methods require considerable time to collect sufficient performance traces representing different resource provisioning possibilities for a target infrastructure to train a useful predictive autoscaling model. In this paper, we tackle this problem by proposing a system that models the response time of microservices through stress testing and then uses a trace-driven simulation to learn a predictive autoscaling model for satisfying response time requirements automatically. The proposed solution reduces the need for collecting performance traces to learn a predictive autoscaling model. Our experimental evaluation on AWS cloud using a microservice under realistic dynamic workloads validates the proposed solution. The validation results show excellent performance to satisfy the response time requirement with only 4.5% extra cost for using the proposed autoscaling method compared to the reactive autoscaling method.