AutoMan: Resource-efficient provisioning with tail latency guarantees for microservices

Modern user-facing services are progressively evolving from large monolithic applications to complex graphs of loosely-coupled microservices. While microservice architecture greatly improves the efficiency of development and operation, it also complicates resource allocation and performance guarantee due to complex dependencies across different microservices. To prevent resource wastage and ensure user satisfaction, we present AutoMan, a learning-driven resource manager for microservices that enables much more efficient resource provisioning while guaranteeing the end-to-end tail latency Service Level Objective (SLO). AutoMan leverages a multi-agent deep deterministic policy gradient (MADDPG)-based method to capture the dependencies across different microservices and to allocate a proper amount of resources to each microservice subject to the target end-to-end tail latency SLO. During runtime, it further proactively identifies the critical microservices responsible for performance anomaly by deriving partial SLOs mathematically, and performs dynamic reprovisioning to mitigate the potential SLO violations. Testbed experiments show that AutoMan can save CPU and memory resources by up to 49.6% and 29.1% on average, while guaranteeing the same end-to-end tail latency objective.(c) 2023 Elsevier B.V. All rights reserved.