Anomaly Detection for Cloud Systems with Dynamic Spatiotemporal Learning

被引:1
作者
Yu, Mingguang [1 ,2 ]
Zhang, Xia [1 ,2 ]
机构
[1] Northeastern Univ, Sch Comp Sci & Engn, Shenyang 110169, Peoples R China
[2] Neusoft Corp, Shenyang 110179, Peoples R China
关键词
System maintenance; anomaly detection; GCN; LSTM; AIOps;
D O I
10.32604/iasc.2023.038798
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
As cloud system architectures evolve continuously, the interac-tions among distributed components in various roles become increasingly complex. This complexity makes it difficult to detect anomalies in cloud systems. The system status can no longer be determined through individual key performance indicators (KPIs) but through joint judgments based on syn-ergistic relationships among distributed components. Furthermore, anomalies in modern cloud systems are usually not sudden crashes but rather grad-ual, chronic, localized failures or quality degradations in a weakly available state. Therefore, accurately modeling cloud systems and mining the hidden system state is crucial. To address this challenge, we propose an anomaly detection method with dynamic spatiotemporal learning (AD-DSTL). AD-DSTL leverages the spatiotemporal dynamics of the system to train an end -to-end deep learning model driven by data from system monitoring to detect underlying anomalous states in complex cloud systems. Unlike previous work that focuses on the KPIs of separate components, AD-DSTL builds a model for the entire system and characterizes its spatiotemporal dynamics based on graph convolutional networks (GCN) and long short-term memory (LSTM). We validated AD-DSTL using four datasets from different backgrounds, and it demonstrated superior robustness compared to other baseline algorithms. Moreover, when raising the target exception level, both the recall and precision of AD-DSTL reached approximately 0.9. Our experimental results demon-strate that AD-DSTL can meet the requirements of anomaly detection for complex cloud systems.
引用
收藏
页码:1787 / 1806
页数:20
相关论文
共 33 条
  • [1] AD-DSTL, 2022, US
  • [2] Aggarwal A., 2020, J COMPUT THEOR NANOS, V17, P4593, DOI [10.1166/jctn.2020.9285, DOI 10.1166/JCTN.2020.9285]
  • [3] A Review of Local Outlier Factor Algorithms for Outlier Detection in Big Data Streams
    Alghushairy, Omar
    Alsini, Raed
    Soule, Terence
    Ma, Xiaogang
    [J]. BIG DATA AND COGNITIVE COMPUTING, 2021, 5 (01) : 1 - 24
  • [4] A robust SVM-based approach with feature selection and outliers detection for classification problems
    Baldomero-Naranjo, Marta
    Martinez-Merino, Luisa I.
    Rodriguez-Chia, Antonio M.
    [J]. EXPERT SYSTEMS WITH APPLICATIONS, 2021, 178
  • [5] Edge importance in a network via line graphs and the matrix exponential
    Cabrera, Omar
    Matar, Mona
    Reichel, Lothar
    [J]. NUMERICAL ALGORITHMS, 2020, 83 (02) : 807 - 832
  • [6] Ranking Causal Anomalies for System Fault Diagnosis via Temporal and Dynamical Analysis on Vanishing Correlations
    Cheng, Wei
    Ni, Jingchao
    Zhang, Kai
    Chen, Haifeng
    Jiang, Guofei
    Shi, Yu
    Zhang, Xiang
    Wang, Wei
    [J]. ACM TRANSACTIONS ON KNOWLEDGE DISCOVERY FROM DATA, 2017, 11 (04)
  • [7] Deeplearning4j, 2022, US
  • [8] Ding K, 2019, Data Min, P594
  • [9] DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
    Du, Min
    Li, Feifei
    Zheng, Guineng
    Srikumar, Vivek
    [J]. CCS'17: PROCEEDINGS OF THE 2017 ACM SIGSAC CONFERENCE ON COMPUTER AND COMMUNICATIONS SECURITY, 2017, : 1285 - 1298
  • [10] Unsupervised Anomaly Detection With LSTM Neural Networks
    Ergen, Tolga
    Kozat, Suleyman Serdar
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2020, 31 (08) : 3127 - 3141