A geometric density-based sample reduction method

被引:0
|
作者
Mohammadi, Mahdi [1 ]
Raahemi, Bijan [1 ]
Akbari, Ahmad [2 ]
机构
[1] Univ Ottawa, Knowledge Discovery & Data Min Lab, Ottawa, ON K1N 6N5, Canada
[2] Iran Univ Sci & Technol, Dept Comp Engn, Tehran, Iran
关键词
Sample reduction; clustering; classification; density-based; membership function;
D O I
10.3233/IDA-150780
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Analysis of network traffic, financial transactions, and mobile communications are examples of applications where examining entire samples of a large dataset is computationally expensive, and requires significant memory space. A common approach to address this challenge is to reduce the number of samples without compromising the accuracy of analyzing them. In this paper, we propose a new cluster-based sample reduction method which is unsupervised, geometric, and density-based. The original data is initially divided into clusters, and each cluster is divided into "portions" defined as the areas between two concentric circles. Then, using the proposed geometric-based formulas, the selection value of each sample belonging to a specific portion is calculated. Samples are then selected from the original data according to the corresponding calculated selection value. The performance of the proposed method is measured on various datasets and compared with several cluster-based and density-based methods. We conduct various experiments on the NSL-KDD, KDDCup99, and IUSTsip datasets, and evaluate the performance of the proposed method by measuring the cluster validity indices, as well as the accuracy of the classifier applied on the reduced data. We demonstrate that the reduced dataset has similar sample scattering as that of the original dataset. We also demonstrate that, while reducing the sample size of the input dataset in half, the classification accuracy is not reduced significantly, indicating that the proposed method selects the most relevant samples from the original dataset.
引用
收藏
页码:1233 / 1257
页数:25
相关论文
共 50 条
  • [31] Data reduction for SVM training using density-based border identification
    Shalaby, Mohammed
    Farouk, Mohamed
    Khater, Hatem A.
    PLOS ONE, 2024, 19 (04):
  • [32] Reduction of Workflow Resource Consumption Using a Density-based Clustering Model
    Zhang, Qimin
    Kremer-Herman, Nathaniel
    Tovar, Benjamin
    Thain, Douglas
    PROCEEDINGS OF WORKS 2018: 13TH IEEE/ACM WORKSHOP ON WORKFLOWS IN SUPPORT OF LARGE-SCALE SCIENCE (WORKS), 2018, : 1 - 9
  • [33] A systematic density-based clustering method using anchor points
    Wang, Yizhang
    Wang, Di
    Pang, Wei
    Miao, Chunyan
    Tan, Ah-Hwee
    Zhou, You
    NEUROCOMPUTING, 2020, 400 : 352 - 370
  • [34] A wavelet thresholding density-based adaptive importance samping method
    Dai, H. (hzdai@hit.edu.cn), 1600, Chinese Society of Theoretical and Applied Mechanics (46):
  • [36] Novel heuristic density-based method for community detection in networks
    Gong, Maoguo
    Liu, Jie
    Ma, Lijia
    Cai, Qing
    Jiao, Licheng
    PHYSICA A-STATISTICAL MECHANICS AND ITS APPLICATIONS, 2014, 403 : 71 - 84
  • [37] A density-based matrix transformation clustering method for electrical load
    Li, Naiwen
    Wu, Xian
    Dong, Jianjun
    Zhang, Dan
    Gao, Shuai
    PLOS ONE, 2022, 17 (08):
  • [38] DBRS: A density-based spatial clustering method with random sampling
    Wang, X
    Hamilton, HJ
    ADVANCES IN KNOWLEDGE DISCOVERY AND DATA MINING, 2003, 2637 : 563 - 575
  • [39] SPHYNX: an accurate density-based SPH method for astrophysical applications
    Cabezon, R. M.
    Garcia-Senz, D.
    Figueira, J.
    ASTRONOMY & ASTROPHYSICS, 2017, 606
  • [40] EDDS: An Enhanced Density-based Method for Clustering Data Streams
    Al Abd Alazeez, Ammar
    Jassim, Sabah
    Du, Hongbo
    2017 46TH INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING WORKSHOPS (ICPPW), 2017, : 103 - 112