Empirical Evaluation of Autoencoder Models for Anomaly Detection in Packet-based NIDS

Abstract

Anomaly detection is critical for network security. Unsupervised learning models trained on benign network traffic data aim to detect anomalies without relying on attack data sets. Autoencoder-based models have emerged as a promising approach for detecting anomalies in network intrusion data. While autoencoder models have predominantly been utilized in flow-based approaches, which are suitable for offline analysis, there is a notable gap in research concerning unsupervised learning, particularly autoencoder-based techniques, for packetbased network intrusion detection. Packet-based network intrusion detection systems (NIDS) enable real-time detection at a granular level, making this area of investigation crucial. In this work, we compare autoencoder models for anomaly detection in packet-based NIDS. A methodological framework is presented for implementing an autoencoder-based network intrusion detection mechanism with packet data. A novel reconstruction error metric is proposed for autoencoders, which is evaluated at different threshold levels to compare the detection accuracies of network traffic anomalies. The effectiveness of autoencoder models is demonstrated on various network attacks and adversarial samples obtained from publicly available network intrusion data sets. The analysis highlights the strengths and limitations of different autoencoders for network traffic anomaly detection. The insights obtained from the empirical evaluation offer valuable guidance to researchers and practitioners aiming to develop an autoencoderbased network intrusion detection mechanism.