Catch the moment: maintaining closed frequent itemsets over a data stream sliding window

被引:82
作者
Chi, Yun [1 ]
Wang, Haixun
Yu, Philip S.
Muntz, Richard R.
机构
[1] Univ Calif Los Angeles, Dept Comp Sci, Los Angeles, CA 90095 USA
[2] IBM Corp, Thomas J Watson Res Ctr, Hawthorne, NY 10532 USA
基金
美国国家科学基金会;
关键词
data streams; sliding window; closed frequent itemset; incremental learning;
D O I
10.1007/s10115-006-0003-0
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This paper considers the problem of mining closed frequent itemsets over a data stream sliding window using limited memory space. We design a synopsis data structure to monitor transactions in the sliding window so that we can output the current closed frequent itemsets at any time. Due to time and memory constraints, the synopsis data structure cannot monitor all possible itemsets. However, monitoring only frequent itemsets will make it impossible to detect new itemsets when they become frequent. In this paper, we introduce a compact data structure, the closed enumeration tree (CET), to maintain a dynamically selected set of itemsets over a sliding window. The selected itemsets contain a boundary between closed frequent itemsets and the rest of the itemsets. Concept drifts in a data stream are reflected by boundary movements in the CET. In other words, a status change of any itemset (e.g., from non-frequent to frequent) must occur through the boundary. Because the boundary is relatively stable, the cost of mining closed frequent itemsets over a sliding window is dramatically reduced to that of mining transactions that can possibly cause boundary movements in the CET. Our experiments show that our algorithm performs much better than representative algorithms for the sate-of-the-art approaches.
引用
收藏
页码:265 / 294
页数:30
相关论文
共 22 条
  • [1] Agarwal R., 1994, P 20 INT C VER LARG, V487, P499
  • [2] A tree projection algorithm for generation of frequent item sets
    Agarwal, RC
    Aggarwal, CC
    Prasad, VVV
    [J]. JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING, 2001, 61 (03) : 350 - 371
  • [3] [Anonymous], 2000, P 2000 ACM SIGMOD IN
  • [4] ASAI T, 2002, P 2002 INT C DAT MIN
  • [5] BAYARDO R, 1998, P ACM SIGMOD
  • [6] CHANG JH, 2003, P 2003 INT C KNOWL D
  • [7] Charikar M., 2002, P 29 INT C AUT LANG
  • [8] CHEUNG D, 1996, P 12 INT C DAT ENG
  • [9] CHEUNG DW, 1997, P 5 INT C DAT SYST A
  • [10] Giannella C, 2003, TR587 IND U