Variational autoencoder-based outlier detection for high-dimensional data

被引:9
作者
Li, Yongmou [1 ,2 ]
Wang, Yijie [1 ,2 ]
Ma, Xingkong [2 ]
机构
[1] Natl Univ Def Technol, Natl Lab Parallel & Distributed Proc, Changsha 410073, Hunan, Peoples R China
[2] Natl Univ Def Technol, Coll Comp, Changsha 410073, Hunan, Peoples R China
基金
国家教育部科学基金资助; 中国国家自然科学基金;
关键词
Variational autoencoders; outlier detection; high-dimensional data;
D O I
10.3233/IDA-184240
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Analysis of high-dimensional data often suffers from the curse of dimensionality and the complicated correlation among dimensions. Dimension reduction methods often are used to alleviate these problems. Existing outlier detection methods based on dimension reduction usually only rely on reconstruction error to detect outlier or apply conventional outlier detection methods to the reduced data, which could deteriorate the performance of outlier detection as only considering part of the information from data. Few studies have been done to combine these two strategies to do outlier detection. In this paper, we proposed an outlier detection method based on Variational Autoencoder (VAE), which combines low-dimensional representation and reconstruction error to detect outliers. Specifically, we first model the data use VAE, then extract four outlier scores from VAE model, finally propose an ensemble method to combine the four outlier scores. The experiments conducted on six real-world datasets show that the proposed method performs better than or at least comparable to state of the art methods.
引用
收藏
页码:991 / 1002
页数:12
相关论文
共 50 条
  • [21] High-dimensional outlier detection using random projections
    Navarro-Esteban, P.
    Cuesta-Albertos, J. A.
    TEST, 2021, 30 (04) : 908 - 934
  • [22] Sparse signal shrinkage and outlier detection in high-dimensional quantile regression with variational Bayes
    Lim, Daeyoung
    Park, Beomjo
    Nott, David
    Wang, Xueou
    Choi, Taeryon
    STATISTICS AND ITS INTERFACE, 2020, 13 (02) : 237 - 249
  • [23] Unsupervised Artificial Neural Networks for Outlier Detection in High-Dimensional Data
    Popovic, Daniel
    Fouche, Edouard
    Boehm, Klemens
    ADVANCES IN DATABASES AND INFORMATION SYSTEMS, ADBIS 2019, 2019, 11695 : 3 - 19
  • [24] Outlier detection based on variance of angle in high dimensional data
    Liu, Wenting
    SIXTH INTERNATIONAL CONFERENCE ON ELECTRONICS AND INFORMATION ENGINEERING, 2015, 9794
  • [25] Support high-order tensor data description for outlier detection in high-dimensional big sensor data
    Deng, Xiaowu
    Jiang, Peng
    Peng, Xiaoning
    Mi, Chunqiao
    FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2018, 81 : 177 - 187
  • [26] Outlier detection in high-dimensional regression model
    Wang, Tao
    Li, Zhonghua
    COMMUNICATIONS IN STATISTICS-THEORY AND METHODS, 2017, 46 (14) : 6947 - 6958
  • [27] Manifold-based denoising, outlier detection, and dimension reduction algorithm for high-dimensional data
    Zhao, Guanghua
    Yang, Tao
    Fu, Dongmei
    INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2023, 14 (11) : 3923 - 3942
  • [28] Manifold-based denoising, outlier detection, and dimension reduction algorithm for high-dimensional data
    Guanghua Zhao
    Tao Yang
    Dongmei Fu
    International Journal of Machine Learning and Cybernetics, 2023, 14 : 3923 - 3942
  • [29] MSS-PAE: Saving Autoencoder-based Outlier Detection from Unexpected Reconstruction
    Tan, Xu
    Yang, Jiawei
    Chen, Junqi
    Rahardja, Sylwan
    Rahardja, Susanto
    PATTERN RECOGNITION, 2025, 163
  • [30] Anomaly detection for high-dimensional data using a novel autoencoder-support vector machine
    Jiang, Zhuo
    Huang, Xiao
    Wang, Rongbin
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2023, 45 (06) : 9457 - 9469