LogSpy: System Log Anomaly Detection for Distributed Systems

Log analysis is an important part of distributed system management. System log records the running status of the system and contains a lot of important and valuable information. This paper proposes an anomaly detection method, LogSpy, for distributed systems. It uses the combination of natural language processing technology and clustering algorithm for log template mining and feature extraction. In anomaly detection, it is found that there are a large number of remote calls in the distributed systems and traditional CNN has certain limitations on this small amount of negative sample data. LogSpy introduces the attention mechanism in detection algorithm and optimizes the detection window and computational complexity. Experiments conducted on the OpenStack test platform show that LogSpy can perform excellent anomaly detection on distributed systems compared to traditional anomaly detection methods.