LogKG: Log Failure Diagnosis Through Knowledge Graph

被引:4
作者
Sui, Yicheng [1 ]
Zhang, Yuzhe [1 ]
Sun, Jianjun [2 ]
Xu, Ting [1 ]
Zhang, Shenglin [3 ,4 ,5 ]
Li, Zhengdan [1 ]
Sun, Yongqian [1 ]
Guo, Fangrui [6 ]
Shen, Junyu [1 ]
Zhang, Yuzhi [3 ,4 ,5 ]
Pei, Dan [7 ,8 ]
Yang, Xiao [2 ]
Yu, Li [2 ]
机构
[1] Nankai Univ, Tianjin 300071, Peoples R China
[2] China Mobile Commun Corp, Beijing 100032, Peoples R China
[3] Nankai Univ, Coll Software, Tianjin 300071, Peoples R China
[4] Minist Educ, Key Lab Data & Intelligent Syst Secur, Tianjin 300071, Peoples R China
[5] Haihe Lab Informat Technol Applicat Innovat HL IT, Tianjin 300071, Peoples R China
[6] Accumulus Technol China Co Ltd, Tianjin 300392, Peoples R China
[7] Tsinghua Univ, Dept Comp Sci, Beijing 100190, Peoples R China
[8] Beijing Natl Res Ctr Informat Sci & Technol, Beijing 100190, Peoples R China
关键词
Semantics; Task analysis; Sun; Manuals; Security; Natural language processing; Knowledge graphs; Cluster; diagnosis; embedding; LogKG;
D O I
10.1109/TSC.2023.3293890
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Logs are one of the most valuable data to describe the running state of services. Failure diagnosis through logs is crucial for service reliability and security. The current automatic log failure diagnosis methods cannot fully use the multiple fields of logs, which fail to capture the relation between them. In this article, we propose LogKG, a new framework for diagnosing failures based on knowledge graphs (KG) of logs. LogKG fully extracts entities and relations from logs to mine multi-field information and their relations through the KG. To fully use the information represented by KG, we propose a failure-oriented log representation (FOLR) method to extract the failure-related patterns. Utilizing the OPTICS clustering method, LogKG aggregates historical failure cases, labels typical failure cases, and trains a failure diagnosis model to identify the root cause. We evaluate the effectiveness of LogKG on a real-world log dataset and a public log dataset, respectively, showing that it outperforms existing methods. With the deployment in a top-tier global Internet Service Provider (ISP), we demonstrate the performance and practicability of LogKG.
引用
收藏
页码:3493 / 3507
页数:15
相关论文
共 50 条
  • [21] Defining a Knowledge Graph Development Process Through a Systematic Review
    Tamasauskaite, Gyte
    Groth, Paul
    [J]. ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY, 2023, 32 (01)
  • [22] LogGraph: Log Event Graph Learning Aided Robust Fine-Grained Anomaly Diagnosis
    Li, Jiangming
    He, Huasen
    Chen, Shuangwu
    Jin, Dong
    Yang, Jian
    [J]. IEEE TRANSACTIONS ON DEPENDABLE AND SECURE COMPUTING, 2024, 21 (04) : 1876 - 1889
  • [23] Validating Security Requirement Specifications through the use of a Knowledge Graph
    Haar, Lynn Vonder
    Reynolds, Sarah
    Procko, Tyler
    Ochoa, Omar
    [J]. 2023 IEEE 17TH INTERNATIONAL CONFERENCE ON SEMANTIC COMPUTING, ICSC, 2023, : 244 - 251
  • [24] Integrating Entity Attributes for Error-Aware Knowledge Graph Embedding
    Zhang, Qinggang
    Dong, Junnan
    Tan, Qiaoyu
    Huang, Xiao
    [J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2024, 36 (04) : 1667 - 1682
  • [25] Global Graph Attention Embedding Network for Relation Prediction in Knowledge Graphs
    Li, Qian
    Wang, Daling
    Feng, Shi
    Niu, Cheng
    Zhang, Yifei
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, 33 (11) : 6712 - 6725
  • [26] Towards Knowledge-Aware and Deep Reinforced Cross-Domain Recommendation Over Collaborative Knowledge Graph
    Li, Yakun
    Hou, Lei
    Li, Juanzi
    [J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2024, 36 (11) : 7171 - 7187
  • [27] Community Enhanced Knowledge Graph for Recommendation
    He, Zhen-Yu
    Wang, Chang-Dong
    Wang, Jinfeng
    Lai, Jian-Huang
    Tang, Yong
    [J]. IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS, 2024, 11 (05) : 5789 - 5802
  • [28] Continual Learning of Knowledge Graph Embeddings
    Daruna, Angel
    Gupta, Mehul
    Sridharan, Mohan
    Chernova, Sonia
    [J]. IEEE ROBOTICS AND AUTOMATION LETTERS, 2021, 6 (02) : 1128 - 1135
  • [29] Online Updates of Knowledge Graph Embedding
    Fei, Luo
    Wu, Tianxing
    Khan, Arijit
    [J]. COMPLEX NETWORKS & THEIR APPLICATIONS X, VOL 2, 2022, 1016 : 523 - 535
  • [30] A Pattern Driven Graph Ranking Approach to Attribute Extraction for Knowledge Graph
    Yang, Muyun
    Chen, Kehai
    Sun, Shuqi
    Han, Zhongyuan
    Kong, Leilei
    Meng, Qingye
    [J]. IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, 2022, 18 (02) : 1250 - 1259