Rasd: Semantic Shift Detection and Adaptation for Network Intrusion Detection

Network Intrusion Detection Systems (NIDSs) based on Deep Neural Network have demonstrated impressive performance in multi-class, closed-world settings, where training and test data follow the same distribution. However, when deployed in real networks, these systems have a limited ability to detect novel attacks which do not belong to already known classes. In this work, we aim to tackle semantic shift, that is the emergence of unknown classes, by proposing a two-phase approach to detect new classes and integrate them into the classification model, while minimising the need for human intervention. While contrastive learning is a promising techniques to tackle semantic shift, it has high computational cost and it is sensitive to imbalanced data. We propose a novel contrastive learning approach based on synthetic centroids which has low computational cost and is robust to class imbalance, making it suitable for application to NIDS. To integrate the shifted samples in the existing model, we also design a novel adaptation method that combines manual labeling and pseudo-labeling to reduce labeling costs. We evaluate our system, Rasd, on two NIDS datasets, finding it excels in both detection and adaptation. For example Rasd improves on the nearest detection baseline F1-score by 6.83% for IDS 2017 and 19.21% for IDS 2018.