Load Balancing Scheme on the Basis of Huffman Coding for P2P Information Retrieval

被引:0
作者
Kurasawa, Hisashi [1 ]
Takasu, Atsuhiro [2 ]
Adachi, Jun [2 ]
机构
[1] Univ Tokyo, Grad Sch Informat Sci & Technol, Tokyo 1018430, Japan
[2] Natl Inst Informat, Tokyo 1018430, Japan
关键词
peer-to-peer; information retrieval; load balancing; Huffman-coding;
D O I
10.1587/transinf.E92.D.2064
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Although a distributed index on a distributed hash table (DHT) enables efficient document query processing in Peer-to-Peer information retrieval (P2P IR), the index costs a lot to construct and it tends to be an unfair management because of the unbalanced term frequency distribution. We devised a new distributed index, named Huffman-DHT, for P2P IR. The new index uses an algorithm similar to Huffman coding with a modification to the DHT structure based on the term distribution. In a Huffman-DHT, a frequent term is assigned to a short ID and allocated a large space in the node ID space in DHT. Throuth ID management. the Huffman-DHT balances the index registration accesses among peers and reduces load concentrations. Huffman-DHT is the first approach to adapt concepts of coding theory and term frequency distribution to load balancing. We evaluated this approach in experiments using a document collection and assessed its load balancing capabilities in P2P IR. The experimental results indicated that it is most effective when the P2P system consists of about 30,000 nodes and contains many documents. Moreover, we proved that we can construct a Huffman-DHT easily by estimating the probability distribution of the term occurrence from a small number of sample documents.
引用
收藏
页码:2064 / 2072
页数:9
相关论文
共 50 条
  • [21] The Application of Cultural Algorithm in Load Balancing of Hybrid P2P Network
    Fu Xiao-ling
    Zhang Jing
    [J]. THIRD INTERNATIONAL CONFERENCE ON GENETIC AND EVOLUTIONARY COMPUTING, 2009, : 19 - 22
  • [22] A Self-Adaptive Load Balancing Strategy for P2P Grids
    Huang, Po-Jung
    Yu, You-Fu
    Chen, Quan-Jie
    Huang, Tian-Liang
    Lai, Kuan-Chou
    Li, Kuan-Ching
    [J]. ALGORITHMS AND ARCHITECTURES FOR PARALLEL PROCESSING, PT 2, PROCEEDINGS, 2010, 6082 : 338 - +
  • [23] Load Balancing and Range Queries in P2P Systems Using P-Ring
    Crainiceanu, Adina
    Linga, Prakash
    Machanavajjhala, Ashwin
    Gehrke, Johannes
    Shanmugasundaram, Jayavel
    [J]. ACM TRANSACTIONS ON INTERNET TECHNOLOGY, 2011, 10 (04)
  • [24] Efficient, proximity-aware load balancing for DHT-based P2P systems
    Zhu, YW
    Hu, YM
    [J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2005, 16 (04) : 349 - 361
  • [25] Load Balancing Using Load Threshold Adjustment and Incentive Mechanism in Structured P2P Systems
    Bok, Kyoungsoo
    Yoon, Jonghyeon
    Lim, Jongtae
    Yoo, Jaesoo
    [J]. IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2019, E102D (05) : 1093 - 1096
  • [26] A Load-balancing Approach for DHT-based P2P Networks
    Tan, Yunsong
    [J]. PROCEEDINGS OF THE 2009 PACIFIC-ASIA CONFERENCE ON CIRCUITS, COMMUNICATIONS AND SYSTEM, 2009, : 191 - 193
  • [27] Dynamic Load Balancing with Multiple Hash Functions in Structured P2P Systems
    Mu, Yuqi
    Yu, Cuibo
    Ma, Tao
    Zhang, Chunbong
    Zheng, Wei
    Zhang, Xiaohua
    [J]. 2009 5TH INTERNATIONAL CONFERENCE ON WIRELESS COMMUNICATIONS, NETWORKING AND MOBILE COMPUTING, VOLS 1-8, 2009, : 5364 - 5367
  • [28] OBIRE: ontology based bibliographic Information retrieval in P2P networks
    Liu, Xiangyu
    Li, Maozhen
    Liu, Yang
    Qi, Man
    [J]. INTERNATIONAL JOURNAL OF DISTRIBUTED SYSTEMS AND TECHNOLOGIES, 2010, 1 (04) : 58 - 73
  • [29] Scalable content-based ranking in P2P information retrieval
    Puh, Maroje
    Luu, Toan
    Zarko, Ivana Podnar
    Rajman, Martin
    [J]. KNOWLEDGE - BASED INTELLIGENT INFORMATION AND ENGINEERING SYSTEMS, PT 1, PROCEEDINGS, 2008, 5177 : 633 - +
  • [30] Beyond Term Indexing: A P2P Framework for Web Information Retrieval
    Podnar, Ivana
    Rajman, Martin
    Luu, Toan
    Klemm, Fabius
    Aberer, Karl
    [J]. INFORMATICA-JOURNAL OF COMPUTING AND INFORMATICS, 2006, 30 (02): : 153 - 161