Tree and Graph Based Two-Stages Routing for Approximate Nearest Neighbor Search

被引:0
|
作者
Li, Jiannan [1 ]
Zhang, Zhenyu [1 ]
Wang, Xiaoling [1 ]
Li, Haoyang [2 ]
机构
[1] East China Normal Univ, Shanghai, Peoples R China
[2] Montverde Acad Shanghai, Shanghai, Peoples R China
来源
WEB AND BIG DATA, APWEB-WAIM 2024, PT III | 2024年 / 14963卷
基金
国家重点研发计划;
关键词
Approximate nearest neighbor search; Reinforcement learning; Graph Index; Tree Index; Hybrid Index;
D O I
10.1007/978-981-97-7238-4_24
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
With the expansion of extensive datasets of high-dimensional vectors, the approximate nearest neighbor search (ANNS) has become increasingly significant in data mining. Among the prevailing ANNS methods, graph-based methods achieve high recall at the cost of excessive memory usage, whereas tree-based methods are memory-efficient but suffer from low recall and increased search complexity. In this paper, we propose a hybrid index based on Ball tree and Hierarchical Navigable Small World(HNSW). Ball-HNSW employs a two-stages hybrid framework to accelerate the identification of approximate nearest neighbors. For datasets with different distributions, we utilize reinforcement learning to optimize parameter combinations by simulating decisions during the search process, thereby improving search results. Experiments confirm that Ball-HNSW outperforms other state-of-the-art methods on multiple datasets.
引用
收藏
页码:376 / 390
页数:15
相关论文
共 50 条
  • [1] Tree-based compact hashing for approximate nearest neighbor search
    Hou, Guangdong
    Cui, Runpeng
    Pan, Zheng
    Zhang, Changshui
    NEUROCOMPUTING, 2015, 166 : 271 - 281
  • [2] Optimizing Graph-based Approximate Nearest Neighbor Search: Stronger and Smarter
    Liu, Jun
    Zhu, Zhenhua
    Hu, Jingbo
    Sun, Hanbo
    Liu, Li
    Liu, Lingzhi
    Dai, Guohao
    Yang, Huazhong
    Wang, Yu
    2022 23RD IEEE INTERNATIONAL CONFERENCE ON MOBILE DATA MANAGEMENT (MDM 2022), 2022, : 179 - 184
  • [3] Quantization to speedup approximate nearest neighbor search
    Hao Peng
    Neural Computing and Applications, 2024, 36 : 2303 - 2313
  • [4] Competitive Quantization for Approximate Nearest Neighbor Search
    Ozan, Ezgi Can
    Kiranyaz, Serkan
    Gabbouj, Moncef
    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2016, 28 (11) : 2884 - 2894
  • [5] Quantization to speedup approximate nearest neighbor search
    Peng, Hao
    NEURAL COMPUTING & APPLICATIONS, 2024, 36 (05) : 2303 - 2313
  • [6] Relative NN-Descent: A Fast Index Construction for Graph-Based Approximate Nearest Neighbor Search
    Ono, Naoki
    Matsui, Yusuke
    PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 1659 - 1667
  • [7] NDSEARCH: Accelerating Graph-Traversal-Based Approximate Nearest Neighbor Search through Near Data Processing
    Wang, Yitu
    Li, Shiyu
    Zheng, Qilin
    Song, Linghao
    Li, Zongwang
    Chang, Andrew
    Li, Hai ''Helen''
    Chen, Yiran
    2024 ACM/IEEE 51ST ANNUAL INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE, ISCA 2024, 2024, : 368 - 381
  • [8] A Revisit of Hashing Algorithms for Approximate Nearest Neighbor Search
    Cai, Deng
    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2021, 33 (06) : 2337 - 2348
  • [9] Fast spectral analysis for approximate nearest neighbor search
    Jing Wang
    Jie Shen
    Machine Learning, 2022, 111 : 2297 - 2322
  • [10] Scalable Distributed Hashing for Approximate Nearest Neighbor Search
    Cao, Yuan
    Liu, Junwei
    Qi, Heng
    Gui, Jie
    Li, Keqiu
    Ye, Jieping
    Liu, Chao
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2022, 31 : 472 - 484