Search engine mashup: a new approach for building large scale search applications

Huge resources are required to build a practical search engine that can index the whole immense web. Rather than construct all parts in-house, this paper presents the mashup approach for building large scale search applications. In this approach, multiple sets of data streams are combined into a unified user experience. Unlike metasearch engines that only present results from different search engines side by side or just reassemble them, the mashup search engine focuses more on the integral aggregation and comprehensive representation of data from multiple sources (not only search engines). Besides presenting the architecture, this paper also discusses the techniques for implementing such a system based on the Google App Engine and the Yahoo! BOSS Mashup Framework. The performance of mashup search engine is analyzed, and a prototype mashup file search engine is developed to demonstrate the practicability of this mashup approach. This approach is especially useful for those developers who prefer to stand on the shoulders of giants and to present integral useful results rather than crawl and index the whole web. 1553-9105/ Copyright © 2009 Binary Information Press.