ByteGraph: A High-Performance Distributed Graph Database in ByteDance

被引:5
作者
Li, Changji [1 ,2 ]
Chen, Hongzhi [2 ]
Zhang, Shuai [2 ]
Hu, Yingqian [2 ]
Chen, Chao [2 ]
Zhang, Zhenjie [2 ]
Li, Meng [2 ]
Li, Xiangchen [2 ]
Han, Dongqing [2 ]
Chen, Xiaohui [2 ]
Wang, Xudong [2 ]
Zhu, Huiming [2 ]
Fu, Xuwei [2 ]
Wu, Tingwei [2 ]
Tan, Hongfei [2 ]
Ding, Hengtian [2 ]
Liu, Mengjin [2 ]
Wang, Kangcheng [2 ]
Ye, Ting [2 ]
Li, Lei [2 ]
Li, Xin [2 ]
Wang, Yu [2 ]
Zheng, Chenguang [1 ,2 ]
Yang, Hao [2 ]
Cheng, James [1 ]
机构
[1] Chinese Univ Hong Kong, Hong Kong, Peoples R China
[2] ByteDance Inc, Beijing, Peoples R China
来源
PROCEEDINGS OF THE VLDB ENDOWMENT | 2022年 / 15卷 / 12期
关键词
D O I
10.14778/3554821.3554824
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Most products at ByteDance, e.g., TikTok, Douyin, and Toutiao, naturally generate massive amounts of graph data. To efficiently store, query and update massive graph data is challenging for the broad range of products at ByteDance with various performance requirements. We categorize graph workloads at ByteDance into three types: online analytical, transaction, and serving processing, where each workload has its own characteristics. Existing graph databases have different performance bottlenecks in handling these workloads and none can efficiently handle the scale of graphs at ByteDance. We developed ByteGraph to process these graph workloads with high throughput, low latency and high scalability. There are several key designs in ByteGraph that make it efficient for processing our workloads, including edge-trees to store adjacency lists for high parallelism and low memory usage, adaptive optimizations on thread pools and indexes, and geographic replications to achieve fault tolerance and availability. ByteGraph has been in production use for several years and its performance has shown to be robust for processing a wide range of graph workloads at ByteDance.
引用
收藏
页码:3306 / 3318
页数:13
相关论文
共 33 条
  • [1] [Anonymous], 2022, AZ COSM DB
  • [2] [Anonymous], 2022, Neo4J
  • [3] [Anonymous], 2021, AgensGraph
  • [4] [Anonymous], 2007, ONL SERV
  • [5] [Anonymous], 2022, While Language
  • [6] [Anonymous], 2021, JanusGraph
  • [7] [Anonymous], 2022, AL GDB
  • [8] [Anonymous], 2022, AWS NEPT
  • [9] [Anonymous], 2021, ArangoDB
  • [10] Bell Charles, 2014, MYSQL HIGH AVAILABIL, V2nd