CALA: An unsupervised URL-based web page classification system

被引:20
作者
Hernandez, Inma [1 ]
Rivero, Carlos R. [2 ]
Ruiz, David [3 ]
Corchuelo, Rafael [3 ]
机构
[1] Univ Autonoma Chile, Santiago, Chile
[2] Univ Idaho, Dept Comp Sci, Moscow, ID 83844 USA
[3] Univ Seville, ETSI Informat, E-41012 Seville, Spain
关键词
Web page classification; URL classification; URL patterns; Enterprise web information integration; Web page clustering; NAVIGATION; LANGUAGE;
D O I
10.1016/j.knosys.2013.12.019
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Unsupervised web page classification refers to the problem of clustering the pages in a web site so that each cluster includes a set of web pages that can be classified using a unique class. The existing proposals to perform web page classification do not fulfill a number of requirements that would make them suitable for enterprise web information integration, namely: to be based on a lightweight crawling, so as to avoid interfering with the normal operation of the web site, to be unsupervised, which avoids the need for a training set of pre-classified pages, or to use features from outside the page to be classified, which avoids having to download it. In this article, we propose CALA, a new automated proposal to generate URL-based web page classifiers. Our proposal builds a number of URL patterns that represent the different classes of pages in a web site, so further pages can be classified by matching their URLs to the patterns. Its salient features are that it fulfills all of the previous requirements, and it has been validated by a number of experiments using real-world, top-visited web sites. Our validation proves that CALA is very effective and efficient in practice. (C) 2013 Elsevier B.V. All rights reserved.
引用
收藏
页码:168 / 180
页数:13
相关论文
共 57 条
[1]  
[Anonymous], 2005, Advances in Minimum Description Length: Theory and Applications
[2]  
[Anonymous], SIGMOD C BALT MAR
[3]  
[Anonymous], 2009, SYST RES
[4]  
[Anonymous], 2002, P 8 ACM SIGKDD INT C, DOI DOI 10.1145/775047.775110
[5]  
[Anonymous], 2010, P 3 ACM INT C WEB SE
[6]   Automating Web navigation with the WebVCR [J].
Anupam, V ;
Freire, J ;
Kumar, B ;
Lieuwen, D .
COMPUTER NETWORKS, 2000, 33 (1-6) :503-517
[7]  
Arasu A., 2003, P 2003 ACM SIGMOD IN, P337, DOI DOI 10.1145/872757.872799
[8]  
Bar-Yossef Z., 2002, P 11 INT C WORLD WID, P580, DOI DOI 10.1145/511446.511522
[9]   Do Not Crawl in the DUST: Different URLs with Similar Text [J].
Bar-Yossef, Ziv ;
Keidar, Idit ;
Schonfeld, Uri .
ACM TRANSACTIONS ON THE WEB, 2009, 3 (01)
[10]   A Comprehensive Study of Features and Algorithms for URL-Based Topic Classification [J].
Baykan, Eda ;
Henzinger, Monika ;
Marian, Ludmila ;
Weber, Ingmar .
ACM TRANSACTIONS ON THE WEB, 2011, 5 (03)