Measuring the effectiveness of a web meta-search engine: a user centered evaluation

The rapidly increasing size of the World Wide Web (WWW) created great needs in the discipline of Information Retrieval (IR). Initially these needs were covered by web search engines, which gave a good solution as far as the size of the web was manageable. Later on the size of the web and its rhythm of expansion started creating problems for the search engines, which need increasingly more resources to cover the constantly increasing size of the WWW. In order to deal with these problems the solution of meta-search engines emerged. Meta-search engines are (second-order) search mechanisms that produce their results by retrieving and merging search results from first-level search engines. Instead of maintaining indexes and databases they use other search engines and produce their own results. The challenge in a meta-search engine is first the source selection problem, i.e. which search first-level search engines to select to submit the user's query, and, second the merging problem, i.e. how to merge the results returned from individual search engines. In this paper we report the design and development of a meta-search engine and explore its potential by comparing it to first level web search engines.