An Intelligent and Time-Efficient DDoS Identification Framework for Real-Time Enterprise Networks SAD-F: Spark Based Anomaly Detection Framework

Abstract

Enterprise networks face a magnitude of threats that are managed and mitigated with a combination of proprietary and third-party security tools and services. However, the techniques and principles employed by the said tools, techniques and services are quite conventional and lack the rapid evolution, as required to protect against modern, state-of-the-art threats faced, specifically, against distributed denial of service (DDoS) attacks. The lack of efficiency of a network is directly proportional to the number of applications and services it hosts, particularly to protect against external and internal threats. Moreover, the effectiveness of such security mechanisms relies on their independent and proactive approach, which is only as effective as their knowledge of known malware and their attack vectors that becomes obsolete when there is a new malware or a zero-day vulnerability is exploited.This paper presents an intelligent, highly responsive, and scalable security framework for enterprise networks. The proposed framework incorporates Apache Spark Framework for security analytics and accurately identifies anomalies specifically pertaining to DDoS attacks from real-time network traffic by using customised machine learning algorithms meticulously trained against selected feature-set. The results are tested against different scenarios and bench-marked with the results achieved by related studies in similar scenarios.