TOSS: A Topology-based Scheduler for Storm Clusters

被引：3

作者：

Zhou, Yi ^{[1
]}

Liu, Yangyang ^{[2
]}

Zhang, Chaowei ^{[2
]}

Peng, Xiaopu ^{[2
]}

Oin, Xiao ^{[2
]}

机构：

[1] Columbus State Univ, TSYS Sch Comp Sci, 4225 Univ Ave, Columbus, GA 31907 USA

[2] Auburn Univ, Dept Comp Sci & Sgftware Engn, Auburn, AL 36849 USA

来源：

2020 IEEE 34TH INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM WORKSHOPS (IPDPSW 2020) | 2020年

关键词：

D O I：

10.1109/IPDPSW50202.2020.00101

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

Nowadays, our world is undergoing profound challenges in processing a massive amount of data. A handful of computation technologies emerge as a promising computation platform for data intensive processing. Apache Storm is an outstanding open-source platform for large-scale streaming computation, which is widely used in the industry (e.g., Twitter). Performance bottleneck problems encountered in streaming data applications motivate us to investigate scheduling issues in Storm. A key aspect of tuning Storm performance is to decide how to deploy components of a storm application among all available nodes in a cluster. Driven by our observations, we design and implement a new scheduling strategy called TOSS based on application structures. Compared to the existing round-robin scheduler, TOSS not only judiciously handles tight-bind components, but also balances workloads by introducing a self-tuning mechanism in the deployment stage. We conduct experiments by applying two popular and distinct topologies to evaluate the performance of TOSS. The experimental results suggest that TOSS significantly boost the performance of the round-robin scheduler. In particular, TOSS sub-stantially improves the system throughput of Storm while shortens latency of Storm applications.

引用

页码：587 / 596

页数：10

共 29 条

[11]

Dean J, 2004, USENIX ASSOCIATION PROCEEDINGS OF THE SIXTH SYMPOSIUM ON OPERATING SYSTEMS DESIGN AND IMPLEMENTATION (OSDE '04), P137

[12] Parameter tuning for configuring and analyzing evolutionary algorithms [J].

Eiben, A. E. ;

Smit, S. K. .

SWARM AND EVOLUTIONARY COMPUTATION, 2011, 1 (01) :19-31

[13]

Filgueira R, 2016, PROCEEDINGS OF 7TH INTERNATIONAL WORKSHOP ON DATA-INTENSIVE COMPUTING IN THE CLOUDS (DATACLOUD 2016), P1, DOI [10.1109/DataCloud.2016.4, 10.1109/DataCloud.2016.004]

[14]

Garofalakis M, 2016, DATA CENTRIC SYST AP, P1, DOI 10.1007/978-3-540-28608-0

[15]

Hunt P., 2010, P USENIX C USENIX AN, V8, P9

[16]

Intel-hadoop, 2016, STORM BENCHM

[17] Adaptive Cluster Tendency Visualization and Anomaly Detection for Streaming Data [J].

Kumar, Dheeraj ;

Bezdek, James C. ;

Rajasegarar, Sutharshan ;

Palaniswami, Marimuthu ;

Leckie, Christopher ;

Chan, Jeffrey ;

Gubbi, Jayavardhana .

ACM TRANSACTIONS ON KNOWLEDGE DISCOVERY FROM DATA, 2016, 11 (02)

[18] A Real-Time Twitter Trend Analysis and Visualization Framework [J].

Murthy, Jamuna S. ;

Siddesh, G. M. ;

Srinivasa, K. G. .

INTERNATIONAL JOURNAL ON SEMANTIC WEB AND INFORMATION SYSTEMS, 2019, 15 (02) :1-21

[19] R-Storm: Resource-Aware Scheduling in Storm [J].

Peng, Boyang ;

Hosseini, Mohammad ;

Hong, Zhihao ;

Farivar, Reza ;

Campbell, Roy .

PROCEEDINGS OF THE 16TH ANNUAL MIDDLEWARE CONFERENCE, 2015, :149-161

[20] Manipulating the gibberellin response to reduce plant height in Chrysanthemum Morifolium [J].

Petty, LM ;

Thomas, B ;

Jackson, SD ;

Harberd, N .

PROCEEDINGS OF THE 4TH INTERNATIONAL SYMPOSIUM ON IN VITRO CULTURE AND HORTICULTURAL BREEDING, 2001, (560) :87-90

← 1 2 3 →