An Anomaly Detection Algorithm for Microservice Architecture Based on Robust Principal Component Analysis

被引:24
|
作者
Jin, Mingxu [1 ]
Lv, Aoran [1 ]
Zhu, Yuanpeng [1 ]
Wen, Zijiang [2 ]
Zhong, Yubin [2 ]
Zhao, Zexin [1 ]
Wu, Jiang [1 ]
Li, Hejie [1 ]
He, Hanheng [1 ]
Chen, Fengyi [3 ]
机构
[1] South China Univ Technol, Sch Math, Guangzhou 510641, Peoples R China
[2] JOYY Inc, Guangzhou 511442, Peoples R China
[3] Guangdong Ind Polytech, Sch Management Adm, Guangzhou 510330, Peoples R China
来源
IEEE ACCESS | 2020年 / 8卷
基金
中国国家自然科学基金;
关键词
Anomaly detection; Standards; Topology; Machine learning algorithms; Signal processing algorithms; Licenses; Containers; Microservice architecture; root cause analysis; anomaly detection; PCA;
D O I
10.1109/ACCESS.2020.3044610
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Microservice architecture (MSA) is a new software architecture, which divides a large single application and service into dozens of supporting microservices. With the increasingly popularity of MSA, the security issues of MSA get a lot of attention. In this paper, we propose an algorithm for mining causality and the root cause. Our algorithm consists of two parts: invocation chain anomaly analysis based on robust principal component analysis (RPCA) and a single indicator anomaly detection algorithm. The single indicator anomaly detection algorithm is composed of Isolation Forest (IF) algorithm, One-Class Support Vector Machine (SVM) algorithm, Local Outlier Factor (LOF) algorithm, and 3 sigma principle. For general and network time-consuming anomaly in the process of the MSA, we formulate different anomaly time-consuming detection strategies. We select a batch of sample data and three batches of test data of the 2020 International AIOps Challenge to debug our algorithm. According to the scoring criteria of the competition organizers, our algorithm has an average score of 0.8304 (The full score is 1) in the four batches of data. Our proposed algorithm has higher accuracy than some traditional machine learning algorithms in anomaly detection.
引用
收藏
页码:226397 / 226408
页数:12
相关论文
共 50 条
  • [41] Approximate Bayesian Algorithm for Tensor Robust Principal Component Analysis
    Srakar, Andrej
    NEW FRONTIERS IN BAYESIAN STATISTICS, BAYSM 2021, 2022, 405 : 1 - 9
  • [42] Robust principal component analysis via ES-algorithm
    Yaeji Lim
    Yeonjoo Park
    Hee-Seok Oh
    Journal of the Korean Statistical Society, 2014, 43 : 149 - 159
  • [43] Anomaly Detection in Microservice-Based Systems
    Nobre, Joao
    Pires, E. J. Solteiro
    Reis, Arsenio
    APPLIED SCIENCES-BASEL, 2023, 13 (13):
  • [44] Unsupervised Anomaly Detection in Hyperspectral Imaging: Integrating Tensor Robust Principal Component Analysis With Autoencoding Adversarial Networks
    Emoto, Atsuya
    Matsuoka, Ryo
    IEEE ACCESS, 2025, 13 : 21422 - 21433
  • [45] An improved fault detection algorithm based on wavelet analysis and kernel principal component analysis
    Chen, Liang
    Yu, Yang
    Luo, Jie
    Zhao, Yawei
    2010 CHINESE CONTROL AND DECISION CONFERENCE, VOLS 1-5, 2010, : 1723 - 1726
  • [46] Improved principal component analysis for anomaly detection: Application to an emergency department
    Harrou, Fouzi
    Kadri, Farid
    Chaabane, Sondes
    Tahon, Christian
    Sun, Ying
    COMPUTERS & INDUSTRIAL ENGINEERING, 2015, 88 : 63 - 77
  • [47] A ROBUST PRINCIPAL COMPONENT ANALYSIS
    RUYMGAART, FH
    JOURNAL OF MULTIVARIATE ANALYSIS, 1981, 11 (04) : 485 - 497
  • [48] Robust principal component analysis
    Partridge, Matthew
    Jabri, Marwan
    Neural Networks for Signal Processing - Proceedings of the IEEE Workshop, 2000, 1 : 289 - 298
  • [49] Autonomous profile-based anomaly detection system using principal component analysis and flow analysis
    Fernandes, Gilberto, Jr.
    Rodrigues, Joel J. P. C.
    Proenca, Mario Lemes, Jr.
    APPLIED SOFT COMPUTING, 2015, 34 : 513 - 525
  • [50] Anomaly Detection in POSTFIX mail log using Principal Component Analysis
    Cao-Phi Tran
    Duc-Khanh Tran
    PROCEEDINGS OF 2018 10TH INTERNATIONAL CONFERENCE ON KNOWLEDGE AND SYSTEMS ENGINEERING (KSE), 2018, : 107 - 112