TPStream: low-latency and high-throughput temporal pattern matching on event streams

被引:0
作者
Michael Körber
Nikolaus Glombiewski
Andreas Morgen
Bernhard Seeger
机构
[1] University of Marburg,Database Systems Group
来源
Distributed and Parallel Databases | 2021年 / 39卷
关键词
Event processing; Pattern matching; Query optimization; Parallel event processing; Distributed event processing;
D O I
暂无
中图分类号
学科分类号
摘要
Sequential pattern matching to detect a user-defined sequence of conditions on event streams is a key feature in modern event processing systems. However, the sequential nature of event based pattern matching has two major deficiencies. First, it is hardly possible to express complex temporal relationships between situations lasting for periods of time. Because events are equipped with a single timestamp only, the expressible temporal relations are limited to before/after/at the same time. Second, a sequential pattern is mapped to a continuous subsequence of the input stream starting with an arbitrary event, making efficient parallelization a hard problem. In this paper we present TPStream, a novel event processing operator for complex temporal pattern matching on event streams. TPStream first summarizes incoming events to situations lasting for periods of time, before it matches temporal patterns. With situations, temporal patterns can easily be defined based on Allen’s interval algebra. We also show that situation based temporal pattern matching can be efficiently executed in parallel using multiple threads on a single machine or multiple machines in a cluster. Finally, we present adaptive optimization components continuously tuning the execution strategy of TPStream towards the lowest possible result latency with respect to the overall system load. The results of our experimental evaluation show that TPStream is capable of processing high-volume event streams with both low latency and high throughput while outperforming applicable CEP solutions from academia and industry.
引用
收藏
页码:361 / 412
页数:51
相关论文
共 40 条
[1]  
Ali MH(2009)Microsoft CEP server and online behavioral targeting Proc. VLDB Endow. 2 1558-1561
[2]  
Allen JF(1983)Maintaining knowledge about temporal intervals Commun. ACM 26 832-843
[3]  
Babu S(2004)Exploiting k-constraints to reduce memory overhead in continuous queries over data streams ACM Trans. Database Syst. 29 545-580
[4]  
Srivastava U(2015)Apache flink™: stream and batch processing in a single engine IEEE Data Eng. Bull. 38 28-38
[5]  
Widom J(2010)High-performance dynamic pattern matching over disordered streams Proc. VLDB Endow. 3 220-231
[6]  
Carbone P(2012)Low latency complex event processing on parallel hardware J. Parallel Distrib. Comput. 72 205-218
[7]  
Katsifodimos A(2018)Recent advancements in event processing ACM Comput. Surv. 51 33:1-33:36
[8]  
Ewen S(2005)Join operations in temporal databases VLDB J. 14 2-29
[9]  
Markl V(2015)Running out of bindings? Integrating facts and events in linked data stream processing CEUR Workshop Proc. 1488 63-74
[10]  
Haridi S(2006)Exploiting predicate-window semantics over data streams SIGMOD Rec. 35 3-8