An Ontology-Based Crawler for the Semantic Web

We present work in progress on automated and ontology-guided discovery, extraction and mapping of information sources on the Semantic Web. It concerns an ontology-guided focused crawler to discover and match different data sources. We have developed an automated ontology-matcher embedded in the crawler that relates semantic web documents found during the crawl to an initial topic ontology that describes the domain of interest of the crawl. Similarity coefficients resulting from the matching process are used to guide the crawler to the information sources relevant to the modelled domain of interest. In addition, the matching process also provides links between the different data sources, which helps in the integration of this data at a later stage. In this paper, the overall architecture, the various modules and methods actually implemented are presented.