Machine learning approaches to network intrusion detection for contemporary internet traffic

All organizations, be they businesses, governments, infrastructure or utility providers, depend on the availability and functioning of their computers, computer networks and data centers for all or part of their operations. Network intrusion detection systems are the first line of defense that protect computing infrastructure from external attacks. In this study we develop five different Machine Learning classifiers for a number of attacks. We used the CSE-CIC-IDS2018 dataset, developed in a collaborative effort between the Communications Security Establishment and the Canadian Institute for Cybersecurity. It is an extensive network traffic trace dataset that captures multiple attacks and has become available relatively recently. The previous major dataset used for the development of network intrusion detection systems is the KDD Cup'99 dataset, now going on 22 years, which predates mobile computing, Web 2.0/3.0, social media, streaming video and widespread use of SSL. These significant Internet trends of the last two decades demand a reevaluation and redevelopment of intrusion detectors. Prior studies that designed Machine Learning classifiers using the CSE-CIC-IDS2018 dataset use a large and rich set of features, of which at least one is not dataset-invariant. Almost none have explored the appropriateness of using all available features with datasets containing only a few hundred attack class samples. The classifiers developed in this study rely on a justifiable number of features and their performance is reviewed for stability and generalization by reporting not just average performance over 10 fold cross-validation but also the degree of variation from one fold to the next.