From Machine Learning Based Intrusion Detection to Cost Sensitive Intrusion Response

Machine learning (ML) based intrusion detection systems (IDS) are increasingly used to discover abnormal patterns in network data and predict cyberattacks. However, the construction of intrusion response systems (IRS) used to deploy countermeasures and prevent malicious activities is more challenging because they require in-depth understanding of attack patterns, attacker behavior, and the correlation between different types of attacks. Furthermore, IDSs generate a large number of false positives and the confidence with which an attack can be predicted is usually unknown. As a result of these challenges in IDS and IRSs, inappropriate actions may be deployed, which may reduce network performance and users' ability to perform typical tasks. Therefore, the present work proposes an intrusion detection and response method based on the Calibrated Random Forest (CRF) algorithm to overcome the key challenges related to the construction of an efficient IRS. The proposed CRF is used to quantify uncertainty in the prediction of cyberattacks and expresses each attack as a probability distribution. Subsequently, the predicted probabilities are used as confidence scores and integrated with domain expert knowledge for decision making in an IRS. We then use publicly available intrusion detection data sets to test and evaluate the proposed method based on three metrics: log loss, Brier score, and expected calibration error (ECE). Experimental results show that the proposed method makes intrusion response more reasonable and cost-sensitive, and has the ability to manage criticality, integrate domain knowledge and explain model behavior. It also demonstrates that this method provides an effective solution for security analysts in how to appropriately deploy and prioritize actions.