Real-Time Detection of Network Traffic Anomalies in Big Data Environments Using Deep Learning Models

Abstract

In light of the increasing sophistication of cyberattacks and the rapid growth in network traffic, it is essential to detect network traffic anomalies or intrusions as they occur. Manual inspection is inefficient due to the large volume, speed, and variety network traffic data. This paper suggests using deep learning techniques in order to build intelligent models which can detect network traffic anomalies automatically within big data environments. We present a framework for anomaly detection using long-short-term memory models (LSTM) and convolutional neural network (CNN). The models are based on data extracted from packet captures. The models are evaluated on benchmark intrusion datasets as well as a large scale real network traffic dataset. The results show that deep learning models are able to detect anomalies more effectively than traditional shallow learning methods. Models can handle high-volume streaming data with low latency and in real time. To improve detection efficiency, we also propose optimization methods such as model compression and transfer learning. This work shows the effectiveness of deep learning for real-time anomaly detection within big data environments