Evaluating clustering techniques for network intrusion detection

Detecting network intrusions has been a critical yet difficult task in network security. To protect networks, intrusion detection systems aim to identify attacks with a high detection rate and a low false alarm rate. Temporal changes in network Intrusion patterns and characteristics often render existing classification-based Intrusion detection data mining techniques ineffective. To address this challenge, researchers have recently revived unsupervised learning techniques to help identify anomalous behaviors or new attacks. This paper Investigates and compares four centroid-based clustering algorithms - k-means, Mixture-Of-Spherical Gaussians, Self-Organizing Map, and Neural-Gas - for analyzing large network security data sets. A simple but effective self-labeling heuristic Is proposed to detect attack clusters given unlabeled network traffic audit data. Empirical studies using the network security data set from the DARPA 1998 offline intrusion detection project (KDD 1999 Cup) show the feasibility of unsupervised intrusion detection and promising detection results.