Unsupervised Network Intrusion Detection Systems: Detecting the Unknown without Knowledge

Abstract

Traditional Network Intrusion Detection Systems (NIDSs) rely on either specialized signatures of previously seen attacks, or on expensive and difficult to produce labeled traffic datasets for user-profiling to hunt out network attacks. Despite being opposite in nature, both approaches share a common downside: they require the knowledge provided by an external agent, either in terms of signatures or as normal-operation profiles. In this paper we present UNIDS, an Unsupervised Network Intrusion Detection System capable of detecting unknown network attacks without using any kind of signatures, labeled traffic, or training. UNIDS uses a novel unsupervised outliers detection approach based on Sub-Space Clustering and Multiple Evidence Accumulation techniques to pin-point different kinds of network intrusions and attacks such as DoS/DDoS, probing attacks, propagation of worms, buffer overflows, illegal access to network resources, etc. We evaluate UNIDS in three different traffic datasets, including the well-known KDD99 dataset as well as real traffic traces from two operational networks. We particularly show the ability of UNIDS to detect unknown attacks, comparing its performance against traditional misuse-detection-based NIDSs. In addition, we also evidence the supremacy of our outliers detection approach with respect to different previously used unsupervised detection techniques.

Data Mining in Intrusion Detection: A Comparative Study of Methods, Types and Data Sets

“…The performance of the proposed approach is evaluated through experiments using the well-known KDD Cup 1999 data set and further experiments using the honeypot data recently collected fro m Kyoto University. It is shown that the proposed approach significantly increase the detection rate while the false alarm rate remains lo w. Casas P et al [33] has proposed an unsupervised network intrusion detection system. Proposed method was evaluated on three different traffic datasets, including the well-known KDD dataset as well as real t raffic t races fro m two operational networks.…”

Analysis of the Effect of Clustering the Training Data in Naive Bayes Classifier for Anomaly Network Intrusion Detection

“…Training a supervised classifier like Naive Bayes requires labelling the training data. According to P. Casas, J. Mazel, and P. Owezarski [10], this task is time consuming and complex. Increase in training data translates into increase in training cost due to labelling for large multidimensional data set like kdd 99 benchmark dataset.…”

Partial Least Square based Improved Intrusion Detection System

“…Pedro Casas et al [11] detected new attack (unknown) without knowledge of any signature, labelled data or training. For that purpose, unsupervised network IDS use outlier detection method which is based on subspace clustering and multiple evidence accumulation method for various network intrusion and attack.…”