A Real-Time Network Intrusion Detection based on Transformer-LSTM model

This paper introduces a novel approach to network intrusion detection by leveraging the advanced capabilities of deep learning. Traditional intrusion detection systems (IDS) often rely on rule-based mechanisms or require human intervention for feature selection, which limits their effectiveness against sophisticated and evolving cyber threats. To address these challenges, this paper focuses on the development of an innovative deep learning model that combines Transformer and Long Short-Term Memory (LSTM) architectures, aiming to enhance the accuracy and automation of intrusion detection tasks. The uniqueness of our approach lies in the integration of the proposed deep learning model with Apache Kafka, a leading distributed streaming platform. This integration enables the real-time capture and processing of network data, which is vital for the timely detection of malicious activities. Specifically, Network data is captured and stored in CSV file by flow generator, streamed into Kafka for efficient processing, analyzed by deep learning model, detected intrusions transmitted back, and stored in Elasticsearch. This setup not only provides a mechanism for real-time intrusion detection but also supports advanced data visualization and further analysis. By combining the predictive power of deep learning with the real-time data processing capabilities of Kafka, this research paper presents a comprehensive and scalable solution for detecting network intrusions.