Variational autoencoder-based outlier detection for high-dimensional data

被引:9
作者
Li, Yongmou [1 ,2 ]
Wang, Yijie [1 ,2 ]
Ma, Xingkong [2 ]
机构
[1] Natl Univ Def Technol, Natl Lab Parallel & Distributed Proc, Changsha 410073, Hunan, Peoples R China
[2] Natl Univ Def Technol, Coll Comp, Changsha 410073, Hunan, Peoples R China
基金
国家教育部科学基金资助; 中国国家自然科学基金;
关键词
Variational autoencoders; outlier detection; high-dimensional data;
D O I
10.3233/IDA-184240
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Analysis of high-dimensional data often suffers from the curse of dimensionality and the complicated correlation among dimensions. Dimension reduction methods often are used to alleviate these problems. Existing outlier detection methods based on dimension reduction usually only rely on reconstruction error to detect outlier or apply conventional outlier detection methods to the reduced data, which could deteriorate the performance of outlier detection as only considering part of the information from data. Few studies have been done to combine these two strategies to do outlier detection. In this paper, we proposed an outlier detection method based on Variational Autoencoder (VAE), which combines low-dimensional representation and reconstruction error to detect outliers. Specifically, we first model the data use VAE, then extract four outlier scores from VAE model, finally propose an ensemble method to combine the four outlier scores. The experiments conducted on six real-world datasets show that the proposed method performs better than or at least comparable to state of the art methods.
引用
收藏
页码:991 / 1002
页数:12
相关论文
共 50 条
  • [41] Outlier Detection based on Sparse Coding and Neighbor Entropy in High-dimensional Space
    Gu, Ping
    Chow, Meng
    Shao, Siyu
    17TH ACM INTERNATIONAL CONFERENCE ON COMPUTING FRONTIERS 2020 (CF 2020), 2020, : 202 - 207
  • [42] A NOVEL TENSOR ALGEBRAIC APPROACH FOR HIGH-DIMENSIONAL OUTLIER DETECTION UNDER DATA MISALIGNMENT
    Fan, Bo
    Zhang, Zemin
    Aeron, Shuchin
    2016 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2016, : 3628 - 3632
  • [43] Outlier mining based on Variance of Angle technology research in High-Dimensional Data
    Liu, Wenting
    Pan, Ruikai
    2015 10TH INTERNATIONAL CONFERENCE ON INTELLIGENT SYSTEMS AND KNOWLEDGE ENGINEERING (ISKE), 2015, : 598 - 603
  • [44] PCA leverage: outlier detection for high-dimensional functional magnetic resonance imaging data
    Mejia, Amanda F.
    Nebel, Mary Beth
    Eloyan, Ani
    Caffo, Brian
    Lindquist, Martin A.
    BIOSTATISTICS, 2017, 18 (03) : 521 - 536
  • [45] Vibration-Based Outlier Detection on High Dimensional Data
    Xia, Shuyin
    Wang, Guoyin
    Yu, Hong
    Liu, Qun
    Wang, Jin
    INTERNATIONAL JOURNAL ON ARTIFICIAL INTELLIGENCE TOOLS, 2016, 25 (03)
  • [46] A fast outlier detection strategy for distributed high-dimensional data sets with mixed attributes
    Koufakou, Anna
    Georgiopoulos, Michael
    DATA MINING AND KNOWLEDGE DISCOVERY, 2010, 20 (02) : 259 - 289
  • [47] A fast outlier detection strategy for distributed high-dimensional data sets with mixed attributes
    Anna Koufakou
    Michael Georgiopoulos
    Data Mining and Knowledge Discovery, 2010, 20 : 259 - 289
  • [48] A High-dimensional Outlier Detection Algorithm Base on Relevant Subspace
    Gao, Zhipeng
    Zhao, Yang
    Niu, Kun
    Fan, Yidan
    2017 IEEE 15TH INTL CONF ON DEPENDABLE, AUTONOMIC AND SECURE COMPUTING, 15TH INTL CONF ON PERVASIVE INTELLIGENCE AND COMPUTING, 3RD INTL CONF ON BIG DATA INTELLIGENCE AND COMPUTING AND CYBER SCIENCE AND TECHNOLOGY CONGRESS(DASC/PICOM/DATACOM/CYBERSCI, 2017, : 1001 - 1008
  • [49] Outlier Detection Using Structural Scores in a High-Dimensional Space
    Li, Xiaojie
    Lv, Jiancheng
    Yi, Zhang
    IEEE TRANSACTIONS ON CYBERNETICS, 2020, 50 (05) : 2302 - 2310
  • [50] CELOF: Effective and fast memory efficient local outlier detection in high-dimensional data streams
    Chen, Liang
    Wang, Wei
    Yang, Yun
    APPLIED SOFT COMPUTING, 2021, 102