Pushing the Limits in Event Normalisation to Improve Attack Detection in IDS/SIEM Systems

The current state of affairs regarding the way events are logged by IT systems is the source of many problems for the developers of Intrusion Detection Systems (IDS) and Security Information and Event Management (SIEM) systems. These problems stand in the way of the development of more accurate security solutions that draw their results from the data included within the logs they process. This is mainly caused by a lack of standards that can encapsulate all events in a coherent way. As a result, correlating between logs produced by different systems that use different log formats has been difficult and infeasible in many cases. In order to solve the challenges faced by Correlation Based Intrusion Detection Systems, we provide a platform for normalising events(1) into a unified super event loosely based on the Common Event Expression standard (CEE)[1] developed by the Mitre corporation[2]. We show how our solution is able to normalise seemingly unrelated events into a unified format. Additionally, we demonstrate queries that can detect attacks on collections of normalised logs from different sources.