Feature Selection in the Corrected KDD-dataset

Zargari, Shahrzad; Voorhis, Dave

doi:10.1109/eidwt.2012.10

Cited by 26 publications

(22 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In signature based NIDS, a database of existing known attack signatures is compared with the current system activities in order to alert the network administrators [9]. On the contrary, anomaly based NIDS, deals with detecting of unknown attacks in network traffics [10].…”

Section: Introductionmentioning

confidence: 99%

“…The proposed method achieved 91% classification accuracy using only three input features and 99% classification accuracy using 36 input features, while 41 input features achieved 99% classification accuracy. It is important to mention that some of the researchers have been working on KDD'99 dataset samples rather than the complete training dataset due to the size of this dataset [10].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Feature selection in UNSW-NB15 and KDDCUP'99 datasets

Janarthanan

Zargari

2017

2017 IEEE 26th International Symposium on Industrial Electronics (ISIE)

Self Cite

128

View full text Add to dashboard Cite

Abstract-Machine learning and data mining techniques have been widely used in order to improve network intrusion detection in recent years. These techniques make it possible to automate anomaly detection in network traffics. One of the major problems that researchers are facing is the lack of published data available for research purposes. The KDD'99 dataset was used by researchers for over a decade even though this dataset was suffering from some reported shortcomings and it was criticized by few researchers. In 2009, Tavallaee M. et al. proposed a new dataset (NSL-KDD) extracted from the KDD'99 dataset in order to improve the dataset where it can be used for carrying out research in anomaly detection. The UNSW-NB15 dataset is the latest published dataset which was created in 2015 for research purposes in intrusion detection. This research is analysing the features included in the UNSW-NB15 dataset by employing machine learning techniques and exploring significant features (curse of high dimensionality) by which intrusion detection can be improved in network systems. Therefore, the existing irrelevant and redundant features are omitted from the dataset resulting not only faster training and testing process but also less resource consumption while maintaining high detection rates. A subset of features is proposed in this study and the findings are compared with the previous work in relation to features selection in the KDD'99 dataset.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Feature selection in UNSW-NB15 and KDDCUP'99 datasets

Janarthanan

Zargari

2017

2017 IEEE 26th International Symposium on Industrial Electronics (ISIE)

Self Cite

128

View full text Add to dashboard Cite

show abstract

“…Authors in [32] describe how intrusion detection systems categorise network traffic as either an anomaly or normal. Data mining is employed into an intrusion detection system as a method of extracting the huge volumes of data that exist in network traffic for further analysis [14].…”

Section: Data Miningmentioning

confidence: 99%

“…Zargari and Voorhis [32] examine significant features in anomaly detection systems with an aim to apply them to data mining techniques. Identifying some current challenges of obtaining a comprehensive feature set and establishing a system that eradicates redundant and recurring data from the KDD 99 dataset while also keeping the feature set to a minimal size.…”

Section: B Feature Selectionmentioning

confidence: 99%

Data Mining and Intrusion Detection Systems

Dewa¹,

Μαγλαράς²

2016

ijacsa

View full text Add to dashboard Cite

Abstract-The rapid evolution of technology and the increased connectivity among its components, imposes new cyber-security challenges. To tackle this growing trend in computer attacks and respond threats, industry professionals and academics are joining forces in order to build Intrusion Detection Systems (IDS) that combine high accuracy with low complexity and time efficiency. The present article gives an overview of existing Intrusion Detection Systems (IDS) along with their main principles. Also this article argues whether data mining and its core feature which is knowledge discovery can help in creating Data mining based IDSs that can achieve higher accuracy to novel types of intrusion and demonstrate more robust behaviour compared to traditional IDSs.

show abstract

“…Previously, they directly removed nominal features (i.e., protocol_type, service and flag) supposedly to enable linear analysis. Finally, two feature selection methods extracted from Weka (Correlation-Stepwise-based, and Information-Gainbased) are applied in Zargari and Voorhis (2012), reaching the best performance with a subset of 10 features.…”

Section: Related Workmentioning

confidence: 99%

Analysis of network traffic features for anomaly detection

2014

View full text Add to dashboard Cite

Anomaly detection in communication networks provides the basis for the uncovering of novel attacks, misconfigurations and network failures. Resource constraints for data storage, transmission and processing make it beneficial to restrict input data to features that are (a) highly relevant for the detection task and (b) easily derivable from network observations without expensive operations. Removing strong correlated, redundant and irrelevant features also improves the detection quality for many algorithms that are based on learning techniques. In this paper we address the feature selection problem for network traffic based anomaly detection. We propose a multi-stage feature selection method using filters and stepwise regression wrappers. Our analysis is based on 41 widely-adopted traffic features that are presented in several commonly used traffic data sets. With our combined feature selection method we could reduce the original feature vectors from 41 to only 16 features. We tested our results with five fundamentally different classifiers, observing no significant reduction of the detection performance. In order to quantify the practical benefits of our results, we analyzed the costs for generating individual features from standard IP Flow Information Export records, available at many routers. We show that we can eliminate 13 very costly features and thus reducing the computational effort for on-line feature generation from live traffic observations at network nodes.

show abstract

Feature Selection in the Corrected KDD-dataset

Cited by 26 publications

References 14 publications

Feature selection in UNSW-NB15 and KDDCUP'99 datasets

Feature selection in UNSW-NB15 and KDDCUP'99 datasets

Data Mining and Intrusion Detection Systems

Analysis of network traffic features for anomaly detection

Contact Info

Product

Resources

About