Benchmarks for Evaluating Anomaly Based Intrusion Detection Solutions

Abstract

Anomaly-based Intrusion Detection Systems (IDS) have gained increased popularity over time. There are many proposed anomaly-based systems using different Machine Learning (ML) algorithms and techniques, however there is no standard benchmark to compare them based on quantifiable measures. In this paper, we propose a benchmark that measures both accuracy and performance to produce objective metrics that can be used in the evaluation of each algorithm implementation. We then use this benchmark to compare accuracy as well as the performance of four different Anomaly-based IDS solutions based on various ML algorithms. The algorithms include Naive Bayes, Support Vector Machines, Neural Networks, and K-means Clustering. The benchmark evaluation is performed on the popular NSL-KDD dataset. The experimental results show the differences in accuracy and performance between these Anomaly-based IDS solutions on the dataset. The results also demonstrate how this benchmark can be used to create useful metrics for such comparisons.

An evaluation design for comparing netflow based network anomaly detection systems using synthetic malicious traffic

“…In this paper, we propose a novel methodology to compare netflow-based network anomaly detection systems using synthetically generated malicious traffic modeling a propagating computer worm [5]. The novelty of our work lies in the fact that we have used malicious traffic as the focus instead of normal traffic to evaluate NF-NAD systems [6]. For this methodology, we have restricted the malicious traffic to common attacks-distributed denial of service attacks (DDoS) [7] and scanning attacks.…”