Thread-Level CPU and Memory Usage Control of Custom Code in Multi-tenant SaaS

Software-as-a-Service (SaaS) providers commonly support customization of their services to allow them to attract larger tenant bases. The nature of these customizations in practice ranges from anticipated configuration options to sophisticated code extensions. From a SaaS provider viewpoint, the latter category is particularly challenging as it involves executing untrusted tenant custom code in the SaaS production environment. Proper isolation of custom code in turn requires the ability to control the CPU and memory usage of each tenant. In current practice, OS-level virtualization tools such as hypervisors or containers are predominantly used for this purpose. These techniques, however, constrain the number of tenants that a single node can cost-effectively accommodate. In this paper, we present a practical solution for thread-level tenant isolation, vis-a-vis CPU and memory usage in presence of tenant-provided custom code. Both Java Runtime Environment (JRE) bytecode and tenant code are instrumented with usage control checkpoints which, based on data gathered using the Java Resource Consumption Management API (JSR-284), ensures that CPU and memory usage of tenants remain within their Service-level Agreements (SLA) limits. Our experiments show that the tenant accommodation capacity of single node increases 59 times with the proposed solution instead of containers. This scalability improvement comes at the average cost of 0.31 ns performance overhead per control checkpoint.