Exploring Network Optimizations for Large-Scale Graph Analytics

被引:3
作者
Que, Xinyu [1 ]
Checconi, Fabio [1 ]
Petrini, Fabrizio [1 ]
Liu, Xing [1 ]
Buono, Daniele [1 ]
机构
[1] IBM Corp, TJ Watson Res Ctr, Yorktown Hts, NY 10598 USA
来源
PROCEEDINGS OF SC15: THE INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS | 2015年
关键词
Network Performance; Graph Algorithm; POWER7; IH; BlueGene/Q;
D O I
10.1145/2807591.2807661
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Graph analytics are arguably one of the most demanding workloads for high-performance systems and interconnection networks. Graph applications often display all-to-all, fine-grained, high-rate communication patterns that expose the limits of the network protocol stacks. Load and communication imbalance generate hard-to-predict network hot-spots, and may require computational steering due to unpredictable data distributions. In this paper we present a lightweight communication library, implemented "on the metal" of BlueGene/Q and POWER7 IH that we have used to support large-scale graph algorithms up to 96K processing nodes and 6 million threads. With this library we have explored several optimization techniques, including overlapped communication, non-blocking collectives, message aggregation, and computation in the network for special collective communication patterns, such as parallel prefix. The experimental results show significant performance improvements, ranging from 5X to 10X, when compared to equally optimized MPI implementations.
引用
收藏
页数:10
相关论文
共 27 条
[1]  
[Anonymous], P INT C HIGH PERF CO
[2]  
[Anonymous], 2013, Proceedings ACM SIGMOD International Conference Management Data, DOI DOI 10.1145/2463676.2467799
[3]  
[Anonymous], [No title captured]
[4]  
[Anonymous], P INT C HIGH PERF CO
[5]  
[Anonymous], P 2010 ACM SIGMOD IN, DOI [DOI 10.1145/1807167.1807184, 10.1145/1807167.1807184]
[6]  
[Anonymous], 1999, TECH REPORT STANFORD
[7]  
[Anonymous], 2014, OSDI 14
[8]  
[Anonymous], P 2011 INT C HIGH PE
[9]  
[Anonymous], 2012, P 10 USENIX S OP SYS
[10]  
[Anonymous], 2008, CORR