GraphMap: scalable iterative graph processing using NoSQL

被引:0
作者
Sayan Goswami
Ayam Pokhrel
Kisung Lee
Ling Liu
Qi Zhang
Yang Zhou
机构
[1] Louisiana State University,
[2] Louisiana State University,undefined
[3] Georgia Institute of Technology,undefined
[4] IBM Thomas J. Watson Research Center,undefined
[5] Auburn University,undefined
来源
The Journal of Supercomputing | 2020年 / 76卷
关键词
Graph processing; Distributed systems; NoSQL;
D O I
暂无
中图分类号
学科分类号
摘要
Despite having several distributed graph processing frameworks, scalable iterative processing of large graphs is a challenging problem since the graph and intermediate data need a global view of the graph topology in distributed memory. Although some systems support out-of-core iterative computations, they use a single machine and often require fast storage. In this paper, we present a new distributed iterative graph computation framework, called GraphMap, that utilizes a disk-based NoSQL database system for scalable graph processing while ensuring competitive performance. Extensive experiments on several real-world graphs show that GraphMap is more scalable and often faster than existing distributed memory-based systems for various graph processing workloads.
引用
收藏
页码:6619 / 6647
页数:28
相关论文
共 41 条
  • [1] Bu Y(2014)Pregelix: Big(Ger) graph analytics on a dataflow engine Proc VLDB Endow 8 161-172
  • [2] Borkar V(2011)Loop recognition in C++/Java/go/scala Proc Scala Days 2011 38-310
  • [3] Jia J(2017)A distributed multi-GPU system for fast graph processing Proc VLDB Endow 11 297-392
  • [4] Carey MJ(1998)A fast and high quality multilevel scheme for partitioning irregular graphs SIAM J Sci Comput 20 359-1905
  • [5] Condie T(2013)Scaling queries over Big RDF graphs with semantic hash partitioning Proc VLDB Endow 6 1894-727
  • [6] Hundt R(2012)Distributed GraphLab: a framework for machine learning and data mining in the cloud Proc VLDB Endow 5 716-204
  • [7] Jia Z(2013)From “think like a vertex” to “think like a graph Proc VLDB Endow 7 193-111
  • [8] Kwon Y(1990)A bridging model for parallel computation Commun ACM 33 103-114
  • [9] Shipman G(2018)Graphd: distributed vertex-centric graph processing beyond the memory limit IEEE Trans Parallel Distrib Syst 29 99-829
  • [10] McCormick P(2008)Velvet: algorithms for de novo short read assembly using de Bruijn graphs Genome Res 18 821-1273