An Intelligent and Time-Efficient DDoS Identification Framework for Real-Time Enterprise Networks: SAD-F: Spark Based Anomaly Detection Framework

Enterprise networks face a large number of threats that are managed and mitigated with a combination of proprietary and third-party security tools and services. However, the techniques and principles employed by the said tools, processes, and services are quite conventional. They lack the rapid evolution, as required to protect against modern, state-of-the-art threats faced, specifically, against distributed denial of service (DDoS) attacks. The lack of efficiency of a network is directly proportional to the number of applications and services it hosts, mainly to protect against external and internal threats. Moreover, the effectiveness of such security mechanisms relies on their independent and proactive approach, which is useful for known malware and their attack vectors, but become obsolete when there is a new malware or zero-day vulnerability is exploits. This paper presents an intelligent, highly responsive, and scalable security framework for enterprise networks. The proposed framework incorporates Apache Spark Framework for security analytics. It accurately identifies anomalies related to DDoS attacks from real-time network traffic by using customized machine learning algorithms, meticulously trained against selected feature-set. Encouraging results are obtained when tested against different scenarios and bench-marked with the results achieved by related studies in similar scenarios.