A geometric density-based sample reduction method

被引:0
|
作者
Mohammadi, Mahdi [1 ]
Raahemi, Bijan [1 ]
Akbari, Ahmad [2 ]
机构
[1] Univ Ottawa, Knowledge Discovery & Data Min Lab, Ottawa, ON K1N 6N5, Canada
[2] Iran Univ Sci & Technol, Dept Comp Engn, Tehran, Iran
关键词
Sample reduction; clustering; classification; density-based; membership function;
D O I
10.3233/IDA-150780
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Analysis of network traffic, financial transactions, and mobile communications are examples of applications where examining entire samples of a large dataset is computationally expensive, and requires significant memory space. A common approach to address this challenge is to reduce the number of samples without compromising the accuracy of analyzing them. In this paper, we propose a new cluster-based sample reduction method which is unsupervised, geometric, and density-based. The original data is initially divided into clusters, and each cluster is divided into "portions" defined as the areas between two concentric circles. Then, using the proposed geometric-based formulas, the selection value of each sample belonging to a specific portion is calculated. Samples are then selected from the original data according to the corresponding calculated selection value. The performance of the proposed method is measured on various datasets and compared with several cluster-based and density-based methods. We conduct various experiments on the NSL-KDD, KDDCup99, and IUSTsip datasets, and evaluate the performance of the proposed method by measuring the cluster validity indices, as well as the accuracy of the classifier applied on the reduced data. We demonstrate that the reduced dataset has similar sample scattering as that of the original dataset. We also demonstrate that, while reducing the sample size of the input dataset in half, the classification accuracy is not reduced significantly, indicating that the proposed method selects the most relevant samples from the original dataset.
引用
收藏
页码:1233 / 1257
页数:25
相关论文
共 50 条
  • [1] A Clustering Density-Based Sample Reduction Method
    Mohammadi, Mahdi
    Raahemi, Bijan
    Akbari, Ahmad
    ADVANCES IN ARTIFICIAL INTELLIGENCE, CANADIAN AI 2014, 2014, 8436 : 319 - 325
  • [2] Geometric algorithms for density-based data clustering
    Chen, DZ
    Smid, M
    Xu, B
    ALGORITHMS-ESA 2002, PROCEEDINGS, 2002, 2461 : 284 - 296
  • [3] Geometric algorithms for density-based data clustering
    Chen, DZ
    Smid, M
    Xu, B
    INTERNATIONAL JOURNAL OF COMPUTATIONAL GEOMETRY & APPLICATIONS, 2005, 15 (03) : 239 - 260
  • [4] An improved method for density-based clustering
    Jin, Hong
    Wang, Shuliang
    Zhou, Qian
    Li, Ying
    INTERNATIONAL JOURNAL OF DATA MINING MODELLING AND MANAGEMENT, 2014, 6 (04) : 347 - 368
  • [5] An ensemble density-based clustering method
    Xia, Luning
    Jing, Jiwu
    PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON INTELLIGENT SYSTEMS AND KNOWLEDGE ENGINEERING (ISKE 2007), 2007,
  • [6] Enhancing density-based data reduction using entropy
    Huang, D.
    Chow, Tommy W. S.
    NEURAL COMPUTATION, 2006, 18 (02) : 470 - 495
  • [7] A density-based data reduction algorithm for robust estimators
    Ferraz, L.
    Felip, R.
    Martinez, B.
    Binefa, X.
    PATTERN RECOGNITION AND IMAGE ANALYSIS, PT 2, PROCEEDINGS, 2007, 4478 : 355 - +
  • [8] A Density-based Anomaly Detection Method for MapReduce
    Wang, Kai
    Wang, Ying
    Yin, Bo
    2012 11TH IEEE INTERNATIONAL SYMPOSIUM ON NETWORK COMPUTING AND APPLICATIONS (NCA), 2012, : 159 - 162
  • [9] A Density-based Discretization Method With Inconsistency Evaluation
    Zhao, Rong
    Qu, Yanpeng
    Deng, Ansheng
    Zwiggelaar, Reyer
    PROCEEDINGS OF 2018 TENTH INTERNATIONAL CONFERENCE ON ADVANCED COMPUTATIONAL INTELLIGENCE (ICACI), 2018, : 758 - 763
  • [10] Fully Automated Density-Based Clustering Method
    Bataineh, Bilal
    Alzahrani, Ahmad A.
    CMC-COMPUTERS MATERIALS & CONTINUA, 2023, 76 (02): : 1833 - 1851