Tree and Graph Based Two-Stages Routing for Approximate Nearest Neighbor Search

被引:0
作者
Li, Jiannan [1 ]
Zhang, Zhenyu [1 ]
Wang, Xiaoling [1 ]
Li, Haoyang [2 ]
机构
[1] East China Normal Univ, Shanghai, Peoples R China
[2] Montverde Acad Shanghai, Shanghai, Peoples R China
来源
WEB AND BIG DATA, APWEB-WAIM 2024, PT III | 2024年 / 14963卷
基金
国家重点研发计划;
关键词
Approximate nearest neighbor search; Reinforcement learning; Graph Index; Tree Index; Hybrid Index;
D O I
10.1007/978-981-97-7238-4_24
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
With the expansion of extensive datasets of high-dimensional vectors, the approximate nearest neighbor search (ANNS) has become increasingly significant in data mining. Among the prevailing ANNS methods, graph-based methods achieve high recall at the cost of excessive memory usage, whereas tree-based methods are memory-efficient but suffer from low recall and increased search complexity. In this paper, we propose a hybrid index based on Ball tree and Hierarchical Navigable Small World(HNSW). Ball-HNSW employs a two-stages hybrid framework to accelerate the identification of approximate nearest neighbors. For datasets with different distributions, we utilize reinforcement learning to optimize parameter combinations by simulating decisions during the search process, thereby improving search results. Experiments confirm that Ball-HNSW outperforms other state-of-the-art methods on multiple datasets.
引用
收藏
页码:376 / 390
页数:15
相关论文
共 50 条
  • [31] ADAPTIVE BIT ALLOCATION HASHING FOR APPROXIMATE NEAREST NEIGHBOR SEARCH
    Guo, Qin-Zhen
    Zeng, Zhi
    Zhang, Shuwu
    Zhang, Yuan
    Wang, Fangyuan
    2013 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO (ICME 2013), 2013,
  • [32] Principal Component Hashing: An Accelerated Approximate Nearest Neighbor Search
    Matsushita, Yusuke
    Wada, Toshikazu
    ADVANCES IN IMAGE AND VIDEO TECHNOLOGY, PROCEEDINGS, 2009, 5414 : 374 - 385
  • [33] Quantization-Based Approximate Nearest Neighbor Search with Optimized Multiple Residual Codebooks
    Uchida, Yusuke
    Takagi, Koichi
    Kawada, Ryoichi
    IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2011, E94D (07): : 1510 - 1514
  • [34] A Learning-to-Rank Formulation of Clustering-Based Approximate Nearest Neighbor Search
    Vecchiato, Thomas
    Lucchese, Claudio
    Nardini, Franco Maria
    Bruch, Sebastian
    PROCEEDINGS OF THE 47TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, SIGIR 2024, 2024, : 2261 - 2265
  • [35] Optimized residual vector quantization for efficient approximate nearest neighbor search
    Ai, Liefu
    Yu, Junqing
    Wu, Zebin
    He, Yunfeng
    Guan, Tao
    MULTIMEDIA SYSTEMS, 2017, 23 (02) : 169 - 181
  • [36] Self-Organizing Binary Encoding for Approximate Nearest Neighbor Search
    Ozan, Ezgi Can
    Kiranyaz, Serkan
    Gabbouj, Moncef
    Hu, Xiaohua
    2016 24TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO), 2016, : 1103 - 1107
  • [37] Optimized residual vector quantization for efficient approximate nearest neighbor search
    Liefu Ai
    Junqing Yu
    Zebin Wu
    Yunfeng He
    Tao Guan
    Multimedia Systems, 2017, 23 : 169 - 181
  • [38] Binary Hashing for Approximate Nearest Neighbor Search on Big Data: A Survey
    Cao, Yuan
    Qi, Heng
    Zhou, Wenrui
    Kato, Jien
    Li, Keqiu
    Liu, Xiulong
    Gui, Jie
    IEEE ACCESS, 2018, 6 : 2039 - 2054
  • [39] M-PCA Binary Embedding For Approximate Nearest Neighbor Search
    Ozan, Ezgi Can
    Kiranyaz, Serkan
    Gabbouj, Moncef
    2015 IEEE TRUSTCOM/BIGDATASE/ISPA, VOL 2, 2015, : 1 - 5
  • [40] Optimized K-means Hashing for Approximate Nearest Neighbor Search
    Guo, Qin-Zhen
    Zeng, Zhi
    Zhang, Shuwu
    Zhang, Yuan
    Zhang, Guixuan
    MATERIAL SCIENCE, CIVIL ENGINEERING AND ARCHITECTURE SCIENCE, MECHANICAL ENGINEERING AND MANUFACTURING TECHNOLOGY II, 2014, 651-653 : 2168 - 2171