Big data with machine learning enabled intrusion detection with honeypot intelligence system on apache Flink (BDML-IDHIS)

This study introduces BDML-IDHIS, a hybrid intrusion detection system combining artificial neural networks (ANN) and Honeypot intelligence, implemented on Apache Flink for real-time big data processing. The system employs a Message Queuing Telemetry Transport (MQTT) Honeypot integrated with Decision and Redirection Engines to enhance system security Experimental evaluations demonstrate that the proposed model achieves a classification accuracy of 98.09%, significantly outperforming traditional methods such as Support Vector Machine (92.76%) and Random Forest (89.40%). Furthermore, the system's scalability and real-time processing capabilities are validated under varying data sizes, showcasing superior throughput and latency performance compared to Apache Spark-based systems. However, limitations include the computational overhead associated with ANN training and reliance on pre-collected datasets. The study highlights the strengths of the BDML-IDHIS system, including precise attack filtering, real-time processing, and scalability for big data environments. Future work will focus on incorporating feature selection techniques to enhance model efficiency and reduce computational complexity.