An Anomaly Detection Algorithm for Microservice Architecture Based on Robust Principal Component Analysis

被引:24
|
作者
Jin, Mingxu [1 ]
Lv, Aoran [1 ]
Zhu, Yuanpeng [1 ]
Wen, Zijiang [2 ]
Zhong, Yubin [2 ]
Zhao, Zexin [1 ]
Wu, Jiang [1 ]
Li, Hejie [1 ]
He, Hanheng [1 ]
Chen, Fengyi [3 ]
机构
[1] South China Univ Technol, Sch Math, Guangzhou 510641, Peoples R China
[2] JOYY Inc, Guangzhou 511442, Peoples R China
[3] Guangdong Ind Polytech, Sch Management Adm, Guangzhou 510330, Peoples R China
来源
IEEE ACCESS | 2020年 / 8卷
基金
中国国家自然科学基金;
关键词
Anomaly detection; Standards; Topology; Machine learning algorithms; Signal processing algorithms; Licenses; Containers; Microservice architecture; root cause analysis; anomaly detection; PCA;
D O I
10.1109/ACCESS.2020.3044610
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Microservice architecture (MSA) is a new software architecture, which divides a large single application and service into dozens of supporting microservices. With the increasingly popularity of MSA, the security issues of MSA get a lot of attention. In this paper, we propose an algorithm for mining causality and the root cause. Our algorithm consists of two parts: invocation chain anomaly analysis based on robust principal component analysis (RPCA) and a single indicator anomaly detection algorithm. The single indicator anomaly detection algorithm is composed of Isolation Forest (IF) algorithm, One-Class Support Vector Machine (SVM) algorithm, Local Outlier Factor (LOF) algorithm, and 3 sigma principle. For general and network time-consuming anomaly in the process of the MSA, we formulate different anomaly time-consuming detection strategies. We select a batch of sample data and three batches of test data of the 2020 International AIOps Challenge to debug our algorithm. According to the scoring criteria of the competition organizers, our algorithm has an average score of 0.8304 (The full score is 1) in the four batches of data. Our proposed algorithm has higher accuracy than some traditional machine learning algorithms in anomaly detection.
引用
收藏
页码:226397 / 226408
页数:12
相关论文
共 50 条
  • [1] A robust anomaly detection algorithm based on principal component analysis
    Huang, Yingkun
    Jin, Weidong
    Yu, Zhibin
    Li, Bing
    INTELLIGENT DATA ANALYSIS, 2021, 25 (02) : 249 - 263
  • [2] GPR Anomaly Detection with Robust Principal Component Analysis
    Masarik, Matthew P.
    Burns, Joseph
    Thelen, Brian T.
    Kelly, Jack
    Havens, Timothy C.
    DETECTION AND SENSING OF MINES, EXPLOSIVE OBJECTS, AND OBSCURED TARGETS XX, 2015, 9454
  • [3] Anomaly Detection Based on Kernel Principal Component and Principal Component Analysis
    Wang, Wei
    Zhang, Min
    Wang, Dan
    Jiang, Yu
    Li, Yuliang
    Wu, Hongda
    COMMUNICATIONS, SIGNAL PROCESSING, AND SYSTEMS, 2019, 463 : 2222 - 2228
  • [4] Anomaly detection based on kernel principal component and principal component analysis
    Wang, Wei
    Zhang, Min
    Wang, Dan
    Jiang, Yu
    Li, Yuliang
    Wu, Hongda
    Lecture Notes in Electrical Engineering, 2019, 463 : 2222 - 2228
  • [6] Randomized subspace-based robust principal component analysis for hyperspectral anomaly detection
    Sun, Weiwei
    Yang, Gang
    Li, Jialin
    Zhang, Dianfa
    JOURNAL OF APPLIED REMOTE SENSING, 2018, 12 (01)
  • [7] A novel unsupervised anomaly detection based on robust principal component classifier
    Qiu, Wenbin
    Wu, Yu
    Wang, Guoyin
    Yang, Simon X.
    Bai, Jie
    Li, Jieying
    DATA MINING, INTRUSION DETECTION, INFORMATION ASSURANCE, AND DATA NETWORKS SECURITY 2006, 2006, 6241
  • [8] Moving Object Detection Optimization Algorithm Based on Robust Principal Component Analysis
    Yang Yizhong
    Wang Pengfei
    Hu Xionglou
    Wu Nengju
    JOURNAL OF ELECTRONICS & INFORMATION TECHNOLOGY, 2018, 40 (06) : 1309 - 1315
  • [9] Traffic Anomaly Detection Based on Robust Principal Component Analysis Using Periodic Traffic Behavior
    Matsuda, Takahiro
    Morita, Tatsuya
    Kudo, Takanori
    Takine, Tetsuya
    IEICE TRANSACTIONS ON COMMUNICATIONS, 2017, E100B (05) : 749 - 761
  • [10] Adaptive Anomaly Detection in Cloud using Robust and Scalable Principal Component Analysis
    Agrawal, Bikash
    Wiktorski, Tomasz
    Rong, Chunming
    2016 15TH INTERNATIONAL SYMPOSIUM ON PARALLEL AND DISTRIBUTED COMPUTING (ISPDC), 2016, : 100 - 106