Storm @Twitter

被引：579

作者：

Toshniwa, Ankit ^{[1
]}

Taneja, Siddarth ^{[1
]}

Shukla, Amit ^{[1
]}

Ramasamy, Karthik ^{[1
]}

Patel, Jignesh M. ^{[2
]}

Kulkarni, Sanjeev ^{[1
]}

Jackson, Jason ^{[1
]}

Gade, Krishna ^{[1
]}

Fu, Maosong ^{[1
]}

Donham, Jake ^{[1
]}

Bhagat, Nikunj ^{[1
]}

Mittal, Sailesh ^{[1
]}

Ryaboy, Dmitriy ^{[1
]}

机构：

[1] Twitter Inc, San Francisco, CA 94103 USA

[2] Univ Wisconsin, Madison, WI USA

来源：

SIGMOD'14: PROCEEDINGS OF THE 2014 ACM SIGMOD INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA | 2014年

关键词：

D O I：

10.1145/2588555.2595641

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

This paper describes the use of Storm at Twitter. Storm is a real-time fault-tolerant and distributed stream data processing system. Storm is currently being used to run various critical computations in Twitter at scale, and in real-time. This paper describes the architecture of Storm and its methods for distributed scale-out and fault-tolerance. This paper also describes how queries (aka. topologies) are executed in Storm, and presents some operational stories based on running Storm at Twitter. We also present results from an empirical evaluation demonstrating the resilience of Storm in dealing with machine failures. Storm is under active development at Twitter and we also present some potential directions for future work.

引用

页码：147 / 156

页数：10

共 14 条

[1]

Abadi Daniel J., CIDR 2005, P277

[2] MillWheel: Fault-Tolerant Stream Processing at Internet Scale [J].

Akidau, Tyler ;

Balikov, Alex ;

Bekiroglu, Kaya ;

Chernyak, Slava ;

Haberman, Josh ;

Lax, Reuven ;

McVeety, Sam ;

Mills, Daniel ;

Nordstrom, Paul ;

Whittle, Sam .

PROCEEDINGS OF THE VLDB ENDOWMENT, 2013, 6 (11) :1033-1044

[3]

Ali M, 2011, PROC INT CONF DATA, P1242, DOI 10.1109/ICDE.2011.5767878

[4]

Ananthanarayanan Rajagopal, SIGMOD C, P577

[5]

Arasu A., 2003, IEEE DATA ENG B, V26, P19

[6] Retrospective on aurora [J].

Balakrishnan, H ;

Balazinska, M ;

Carney, D ;

Çetintemel, U ;

Cherniack, M ;

Convey, C ;

Galvez, E ;

Salz, J ;

Stonebraker, M ;

Tatbul, N ;

Tibbetts, R ;

Zdonik, S .

VLDB JOURNAL, 2004, 13 (04) :370-383

[7]

Garofalakis Minos N., VLDB 2002

[8]

Graefe Goetz, SIGMOD C, P102

[9]

Hindman B., 2011, 8 USENIX S NETWORKED

[10] Towards a Streaming SQL Standard [J].

Jain, Namit ;

Mishra, Shailendra ;

Srinivasan, Anand ;

Gehrke, Johannes ;

Widom, Jennifer ;

Balakrishnan, Hari ;

Cetintemel, Ugur ;

Cherniack, Mitch ;

Tibbetts, Richard ;

Zdonik, Stan .

PROCEEDINGS OF THE VLDB ENDOWMENT, 2008, 1 (02) :1379-1390

← 1 2 →