Storm @Twitter

被引：572

作者：

Toshniwa, Ankit ^{[1
]}

Taneja, Siddarth ^{[1
]}

Shukla, Amit ^{[1
]}

Ramasamy, Karthik ^{[1
]}

Patel, Jignesh M. ^{[2
]}

Kulkarni, Sanjeev ^{[1
]}

Jackson, Jason ^{[1
]}

Gade, Krishna ^{[1
]}

Fu, Maosong ^{[1
]}

Donham, Jake ^{[1
]}

Bhagat, Nikunj ^{[1
]}

Mittal, Sailesh ^{[1
]}

Ryaboy, Dmitriy ^{[1
]}

机构：

[1] Twitter Inc, San Francisco, CA 94103 USA

[2] Univ Wisconsin, Madison, WI USA

来源：

SIGMOD'14: PROCEEDINGS OF THE 2014 ACM SIGMOD INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA | 2014年

关键词：

D O I：

10.1145/2588555.2595641

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

This paper describes the use of Storm at Twitter. Storm is a real-time fault-tolerant and distributed stream data processing system. Storm is currently being used to run various critical computations in Twitter at scale, and in real-time. This paper describes the architecture of Storm and its methods for distributed scale-out and fault-tolerance. This paper also describes how queries (aka. topologies) are executed in Storm, and presents some operational stories based on running Storm at Twitter. We also present results from an empirical evaluation demonstrating the resilience of Storm in dealing with machine failures. Storm is under active development at Twitter and we also present some potential directions for future work.

引用

页码：147 / 156

页数：10

共 14 条

[1] Abadi Daniel J., CIDR 2005, P277
[2] MillWheel: Fault-Tolerant Stream Processing at Internet Scale
Akidau, Tyler
Balikov, Alex
Bekiroglu, Kaya
Chernyak, Slava
Haberman, Josh
Lax, Reuven
McVeety, Sam
Mills, Daniel
Nordstrom, Paul
Whittle, Sam
[J]. PROCEEDINGS OF THE VLDB ENDOWMENT, 2013, 6 (11): : 1033 - 1044
[3] Ali M, 2011, PROC INT CONF DATA, P1242, DOI 10.1109/ICDE.2011.5767878
[4] Ananthanarayanan Rajagopal, SIGMOD C, P577
[5] Arasu A., 2003, IEEE DATA ENG B, V26, P19
[6] Retrospective on aurora
Balakrishnan, H
Balazinska, M
Carney, D
Çetintemel, U
Cherniack, M
Convey, C
Galvez, E
Salz, J
Stonebraker, M
Tatbul, N
Tibbetts, R
Zdonik, S
[J]. VLDB JOURNAL, 2004, 13 (04) : 370 - 383
[7] Garofalakis Minos N., VLDB 2002
[8] Graefe Goetz, SIGMOD C, P102
[9] Hindman B., 2011, 8 USENIX S NETWORKED
[10] Towards a Streaming SQL Standard
Jain, Namit
Mishra, Shailendra
Srinivasan, Anand
Gehrke, Johannes
Widom, Jennifer
Widom, Jennifer
Balakrishnan, Hari
Cetintemel, Ugur
Cherniack, Mitch
Tibbetts, Richard
Zdonik, Stan
[J]. PROCEEDINGS OF THE VLDB ENDOWMENT, 2008, 1 (02): : 1379 - 1390

← 1 2 →