Intrusion Detection in Industrial Control Systems Based on Deep Reinforcement Learning

被引:6
作者
Sangoleye, Fisayo [1 ]
Johnson, Jay [2 ]
Eleni Tsiropoulou, Eirini [1 ]
机构
[1] Univ New Mexico, Dept Elect & Comp Engn, Albuquerque, NM 87131 USA
[2] DER Secur Corp, Scotts Valley, CA 95066 USA
关键词
Adaptation models; Intrusion detection; Feature extraction; Training; Protocols; Microgrids; Heuristic algorithms; Industrial control; Deep reinforcement learning; Accuracy; SCADA systems; network intrusion detection; SCADA; industrial control systems; microgrids; NETWORKS; SECURITY; MACHINE;
D O I
10.1109/ACCESS.2024.3477415
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
As the threat landscape for Operational Technology (OT) and Supervisory Control and Data Acquisition (SCADA) systems grows more complex, there is a pressing need for intrusion detection systems that can dynamically adapt to evolving attack patterns. Traditional Machine Learning (ML) approaches often require frequent manual retraining and struggle to respond efficiently to these dynamic threats. Deep Reinforcement Learning (DRL) models present a promising solution, offering autonomous learning capabilities, adaptability to diverse scenarios with minimal human intervention, and enhanced intrusion detection for Industrial Control Systems (ICS). This paper presents a novel investigation into the application of various DRL models, including Deep Q-Network (DQN), Double Deep Q-Network (DDQN), Dueling Double Deep Q-Network (D3QN), REINFORCE, Advantage Actor-Critic (A2C), and Proximal Policy Optimization (PPO), for network intrusion detection in ICS. Performance comparisons with traditional ML methods are conducted using relevant metrics. To assess their effectiveness without a live environment, labeled pre-recorded intrusion datasets are utilized, with tailored adaptations for DRL model training outlined. These adaptations include generating data samples in mini-batches, integrating small discount factors, and employing straightforward reward functions. Comprehensive results underscore the efficacy of DRL models in bolstering the detection of advanced cyberattacks within OT environments, surpassing conventional ML approaches.
引用
收藏
页码:151444 / 151459
页数:16
相关论文
共 62 条
[1]  
Abdelkhalek M.., 2022, Iowa State University Der-DNP3 2022 Dataset
[2]   ML-based Anomaly Detection System for DER DNP3 Communication in Smart Grid [J].
Abdelkhalek, Moataz ;
Govindarasu, Manimaran .
2022 IEEE INTERNATIONAL CONFERENCE ON CYBER SECURITY AND RESILIENCE (IEEE CSR), 2022, :209-214
[3]   DeepEdge: A New QoE-Based Resource Allocation Framework Using Deep Reinforcement Learning for Future Heterogeneous Edge-IoT Applications [J].
AlQerm, Ismail ;
Pan, Jianli .
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, 2021, 18 (04) :3942-3954
[4]  
[Anonymous], 2018, Cicflowmeter (Formerly Iscxflowmeter)
[5]  
[Anonymous], 2018, CSECIC-IDS2018
[6]   Deep Reinforcement Adversarial Learning Against Botnet Evasion Attacks [J].
Apruzzese, Giovanni ;
Andreolini, Mauro ;
Marchetti, Mirco ;
Venturi, Andrea ;
Colajanni, Michele .
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, 2020, 17 (04) :1975-1987
[7]  
Barto Andrew., 1998, REINFORCEMENT LEARNI
[8]   Infinite-horizon policy-gradient estimation [J].
Baxter, J ;
Bartlett, PL .
JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH, 2001, 15 :319-350
[9]   Random forests [J].
Breiman, L .
MACHINE LEARNING, 2001, 45 (01) :5-32
[10]   Detecting Phishing Websites through Deep Reinforcement Learning [J].
Chatterjee, Moitrayee ;
Namin, Akbar Siami .
2019 IEEE 43RD ANNUAL COMPUTER SOFTWARE AND APPLICATIONS CONFERENCE (COMPSAC), VOL 2, 2019, :227-232