Benchmarking the robustness of web services

This paper proposes an approach for the evaluation of the robustness of web services, which are complex software components that must provide a robust interface to the client applications. However, although web services are becoming business-critical components , there is no practical way to assess the robustness of the code or to compare alternative implementations concerning robustness. The approach proposed is based on a set of robustness tests (i.e., invalid web services call parameters) that is applied in order to discover both programming and design errors. The web services are classified based on the failures observed during the execution of the tests. The approach is illustrated by evaluating several web services publicly available in the Internet and two different implementations of the web services specified by the standard TPC-App performance benchmark. The proposed approach is useful for both web services providers (to assess the robustness of their web services code) and consumers (to select the web services that best fit their requirements).