Tree and Graph Based Two-Stages Routing for Approximate Nearest Neighbor Search

被引:0
作者
Li, Jiannan [1 ]
Zhang, Zhenyu [1 ]
Wang, Xiaoling [1 ]
Li, Haoyang [2 ]
机构
[1] East China Normal Univ, Shanghai, Peoples R China
[2] Montverde Acad Shanghai, Shanghai, Peoples R China
来源
WEB AND BIG DATA, APWEB-WAIM 2024, PT III | 2024年 / 14963卷
基金
国家重点研发计划;
关键词
Approximate nearest neighbor search; Reinforcement learning; Graph Index; Tree Index; Hybrid Index;
D O I
10.1007/978-981-97-7238-4_24
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
With the expansion of extensive datasets of high-dimensional vectors, the approximate nearest neighbor search (ANNS) has become increasingly significant in data mining. Among the prevailing ANNS methods, graph-based methods achieve high recall at the cost of excessive memory usage, whereas tree-based methods are memory-efficient but suffer from low recall and increased search complexity. In this paper, we propose a hybrid index based on Ball tree and Hierarchical Navigable Small World(HNSW). Ball-HNSW employs a two-stages hybrid framework to accelerate the identification of approximate nearest neighbors. For datasets with different distributions, we utilize reinforcement learning to optimize parameter combinations by simulating decisions during the search process, thereby improving search results. Experiments confirm that Ball-HNSW outperforms other state-of-the-art methods on multiple datasets.
引用
收藏
页码:376 / 390
页数:15
相关论文
共 50 条
  • [21] Flexible product quantization for fast approximate nearest neighbor search
    Jingya Fan
    Yang Wang
    Wenwen Song
    Zhibin Pan
    Multimedia Tools and Applications, 2024, 83 : 53243 - 53261
  • [22] Residual Vector Product Quantization for approximate nearest neighbor search
    Niu, Lushuai
    Xu, Zhi
    Zhao, Longyang
    He, Daojing
    Ji, Jianqiu
    Yuan, Xiaoli
    Xue, Mian
    EXPERT SYSTEMS WITH APPLICATIONS, 2023, 232
  • [23] A fast binary encoding mechanism for approximate nearest neighbor search
    Zhao, Hongwei
    Wang, Zhen
    Liu, Pingping
    Wu, Bin
    NEUROCOMPUTING, 2016, 178 : 112 - 122
  • [24] Adaptive bit allocation hashing for approximate nearest neighbor search
    Guo, Qin-Zhen
    Zeng, Zhi
    Zhang, Shuwu
    NEUROCOMPUTING, 2015, 151 : 719 - 728
  • [25] Product quantization with dual codebooks for approximate nearest neighbor search
    Pan, Zhibin
    Wang, Liangzhuang
    Wang, Yang
    Liu, Yuchen
    NEUROCOMPUTING, 2020, 401 : 59 - 68
  • [26] K-Subspaces Quantization for Approximate Nearest Neighbor Search
    Ozan, Ezgi Can
    Kiranyaz, Serkan
    Gabbouj, Moncef
    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2016, 28 (07) : 1722 - 1733
  • [27] Residual Vector Product Quantization for Approximate Nearest Neighbor Search
    Xu, Zhi
    Niu, Lushuai
    Meng, Ruimin
    Zhao, Longyang
    Ji, Jianqiu
    ADVANCES IN KNOWLEDGE DISCOVERY AND DATA MINING, PAKDD 2022, PT I, 2022, 13280 : 208 - 220
  • [28] Trinary-Projection Trees for Approximate Nearest Neighbor Search
    Wang, Jingdong
    Wang, Naiyan
    Jia, You
    Li, Jian
    Zeng, Gang
    Zha, Hongbin
    Hua, Xian-Sheng
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2014, 36 (02) : 388 - 403
  • [29] Approximate nearest neighbor search by cyclic hierarchical product quantization
    Xu, Zhi
    Zhou, Mengdong
    Liu, Yuxuan
    Zhao, Longyang
    Liu, Jiajia
    SIGNAL IMAGE AND VIDEO PROCESSING, 2025, 19 (06)
  • [30] Approximate Nearest Neighbor Search Using Enhanced Accumulative Quantization
    Ai, Liefu
    Cheng, Hongjun
    Wang, Xiaoxiao
    Chen, Chunsheng
    Liu, Deyang
    Zheng, Xin
    Wang, Yuanzhi
    ELECTRONICS, 2022, 11 (14)