Complex event processing over live archived data streams

With the development of data collection and data processing techniques, event detection has become increasingly vital in application areas such as object-tracking in IOT, network monitoring, financial prediction, and telecommunication consumption mode detection, etc. Event processing is supposed to be completed in one-pass of the data streams which are discarded after pattern matching. Actually, historical streams maintain plentiful information which cannot be simply discarded in many scenarios and some event detection queries are always subscribed over both live and archived (historical) streams. Due to the lackness of event processing over live and archived event streams, this paper addresses key issues of live- archived stream complex event processing. Main works are as follows: (1) Due to large numbers of partial matches generated in a sliding window, partial matches management methods named TPM and STPM are proposed. With STPM, spatial and temporal information are kept into partial matches and the most recent and possible updated partial matches are resided in main memory which can reduce pattern match miss ratio and greatly alleviate external partial match loading I/O cost. (2) Optimization of complex event processing algorithm over live-archived streams based on events selectivity is proposed. (3) Formal cost model of related methods are presented. (4) Based on the proposed partial matches management methods, extensive performance comparison experiments in a prototype CEP system are evaluated(experimental parameters include subwindow size, selectivity, match ratio, hit ratio, etc). Experimental analysis verifies soundness and effectiveness of the proposed methods.