Analytics-Driven Load Testing: An Industrial Experience Report on Load Testing of Large-Scale Systems

Assessing how large-scale software systems behave under load is essential because many problems cannot be uncovered without executing tests of large volumes of concurrent requests. Load-related problems can directly affect the customer-perceived quality of systems and often cost companies millions of dollars. Load testing is the standard approach for assessing how a system behaves under load. However, designing, executing and analyzing a load test can be very difficult due to the scale of the test (e.g., simulating millions of users and analyzing terabytes of data). Over the past decade, we have tackled many load testing challenges in an industrial setting. In this paper, we document the challenges that we encountered and the lessons that we learned as we addressed these challenges. We provide general guidelines for conducting load tests using an analytics-driven approach. We also discuss open research challenges that require attention from the research community. We believe that our experience can be beneficial to practitioners and researchers who are interested in the area of load testing.