Developing internet e-commerce benchmarks

A benchmark is a standard for measuring and comparing the performance of like systems. For new product makers, a benchmark can provide important statistical information so products can be fine-tuned before their deployment. For end users, on the other hand, a benchmark can be used to compare the strengths and weaknesses of different products so that an informed decision can be made about system adoption. Benchmarks aid in estimations of scalability in terms of the number of users and/or transactions that a system can support, and system response times under various loads and hardware/software deployment platforms. This paper focuses on the design issues in developing benchmarks for e-commerce. Because of the multidisciplinary aspects of e-commerce and the various emerging and distinct e-commerce business models, creating a single benchmark for the e-commerce application is not feasible. Add to this the diverse needs of small to medium enterprises (SMEs) and big business and we motivate the need for a benchmark suite for e-commerce. It is the thesis of this paper that the business model plays the primary role in the development of a e-commerce benchmark. It is the business that determines processes and transactions and thus also the database and navigational designs. For illustrative purposes, we step through the design of an e-commerce benchmark specification, WebEC, based on a e-broker (cybermediary) Internet business model. An example implementation of the benchmark specification, based on Microsoft's COM technology, and sample benchmark results are also presented. (C) 1999 Published by Elsevier Science Ltd. All rights reserved.