Outlier detection in transactional data

被引:5
作者
Dash, Manoranjan [1 ]
Lie, Ng Wil [1 ]
机构
[1] Nanyang Technol Univ, Sch Comp Engn, Singapore, Singapore
关键词
Outlier detection; transactional data; epsilon-approximation; sampling;
D O I
10.3233/IDA-2010-0422
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Outlier detection is studied in the context of supervised (with class label) and unsupervised (without class label - e.g., clustering) data. To the best of our knowledge there has been no study on outlier detection in transactional database, e.g. market basket data where each transaction has a number of items and the number of items in each transaction is not constant. Following the definition of outlier by Barnett and Lewis, 1994, we define an outlier transaction in this paper as that which appears to deviate markedly from other transactions of the database in which it occurs. This problem is important, for instance a supermarket manager would like to know the outlier transactions so as to make proper decisions. This problem is not trivial particularly when the number of items is large which is often the case in market basket data. We propose a novel and efficient solution DETACH which is different from other work in this area. The proposed method uses a unique approach of creating a representative sample, and subsequently it determines the degree to which each transaction is an outlier considering the representative sample. The proposed method, unlike its predecessors, does not require any parameter that is hard to set. We test our proposed method using benchmark market basket data from QUEST project. Results show that DETACH is very efficient and accurate.
引用
收藏
页码:283 / 298
页数:16
相关论文
共 29 条
[1]  
AGRAWAL R, 1998, P ACM SIGMOD C MAN D
[2]  
AGRAWAL R, 1994, P 20 VLDB C
[3]  
Alon Noga, 1992, The Probabilistic Method
[4]  
[Anonymous], 1994, Wiley series in probability and mathematical statistics applied probability and statistics
[5]  
[Anonymous], 2001, P ACM CSS WORKSH DAT
[6]  
[Anonymous], 1999, KDD, DOI [10.1145/312129.312195, DOI 10.1016/J.EC0LENG.2010.11.031]
[7]  
[Anonymous], 2004, ARTIFICIAL INTELLIGE
[8]   NOVELTY DETECTION AND NEURAL-NETWORK VALIDATION [J].
BISHOP, CM .
IEE PROCEEDINGS-VISION IMAGE AND SIGNAL PROCESSING, 1994, 141 (04) :217-222
[9]   LOF: Identifying density-based local outliers [J].
Breunig, MM ;
Kriegel, HP ;
Ng, RT ;
Sander, J .
SIGMOD RECORD, 2000, 29 (02) :93-104
[10]  
Cadez I. V., 2001, KDD-2001. Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, P37, DOI 10.1145/502512.502523