A Declarative Environment for Automatic Performance Evaluation in IaaS Clouds

One of the main challenges faced by users of infrastructure-as-a-service (IaaS) clouds is the difficulty to adequately estimate the virtual resources necessary for their applications. Although many cloud providers offer programatic ways to rapidly acquire and release resources, it is important that users have a prior understanding of the impact that each virtual resource type offered by the provider may impose on application performance. This paper presents Cloud Crawler, a new declarative environment aimed at supporting users in describing and automatically executing application performance tests in IaaS clouds. To this end, the environment provides a novel declarative domain-specific language, called Crawl, which supports the description of a variety of performance evaluation scenarios in multiple IaaS clouds; and an extensible Java-based cloud execution engine, called Crawler, which automatically configures, executes and collects the results of each performance evaluation scenario described in Crawl. To illustrate Cloud Crawler's potential benefits, the paper reports on an experimental evaluation of a social network application in two public IaaS cloud providers, in which the proposed environment has successfully been used to investigate the application performance for different virtual machine configurations and under different demand levels.