Testing Intrusion Detection Systems: An engineered approach

The enhancements of Intrusion Detection Systems (IDS) are still bellow expectations. The great number of false positives (false alarms) and false negatives (undetected intrusions) has survived in recent versions as well as in the old ones. This may be -in part- caused by the shortage of an effective, unbiased evaluation and testing methodology that is both scientifically rigorous and technically feasible. The complexity of the environments where Intrusion detection systems operate, makes the evaluation process itself a nontrivial task. For this reason, ad-hoc evaluations often produce results that don't correspond to real world. In this paper, we propose a framework for evaluating IDSes as well as some new metrics. This systematic methodology follows an engineered approach to manage the complexity of the evaluation process and takes into account both environment and IDS characteristics. 514-072.pdf