Identifying important characteristics in the KDD99 intrusion detection dataset by feature selection using a hybrid approach

Araújo, Nelcileno; Oliveira, Ruy de; Ferreira, Ed`Wilson Tavares; Shinoda, Aílton Akira; Bhargava, Bharat

doi:10.1109/ictel.2010.5478852

Cited by 33 publications

(15 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Analysis of KDD-99 dataset with its 41 features was presented based on calculating the information gain and the entropy of each feature to measure its relevance [3]. The fourth feature selection method is proposed by [4]. In this study, a hybrid approach is used to obtain the optimal set of 14 features.…”

Section: Experiments and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Machine-Learning-Based Feature Selection Techniques for Large-Scale Network Intrusion Detection

Al-Jarrah

Siddiqui

ElSalamouny

et al. 2014

2014 IEEE 34th International Conference on Distributed Computing Systems Workshops

View full text Add to dashboard Cite

Nowadays, we see more and more cyber-attacks on major Internet sites and enterprise networks. Intrusion Detection System (IDS) is a critical component of such infrastructure defense mechanism. IDS monitors and analyzes networks' activities for potential intrusions and security attacks. Machinelearning (ML) models have been well accepted for signaturebased IDSs due to their learnability and flexibility. However, the performance of existing IDSs does not seem to be satisfactory due to the rapid evolution of sophisticated cyber threats in recent decades. Moreover, the volumes of data to be analyzed are beyond the ability of commonly used computer software and hardware tools. They are not only large in scale but fast in/out in terms of velocity. In big data IDS, the one must find an efficient way to reduce the size of data dimensions and volumes. In this paper, we propose novel feature selection methods, namely, RF-FSR (RandomForest-Forward Selection Ranking) and RF-BER (RandomForest-Backward Elimination Ranking). The features selected by the proposed methods were tested and compared with three of the most well-known feature sets in the IDS literature. The experimental results showed that the selected features by the proposed methods effectively improved their detection rate and false-positive rate, achieving 99.8% and 0.001% on well-known KDD-99 dataset, respectively.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

“…The proposed methods use an ensemble of Random Forest (RF) algorithm, with forward and backward ranking features selection techniques [1][2]. To prove the usefulness of the proposed methods, we compare our results with those of other three well-known feature sets [3][4][5] on KDD-99 dataset.…”

Section: Introductionmentioning

confidence: 94%

Machine-Learning-Based Feature Selection Techniques for Large-Scale Network Intrusion Detection

Al-Jarrah

Siddiqui

ElSalamouny

et al. 2014

2014 IEEE 34th International Conference on Distributed Computing Systems Workshops

View full text Add to dashboard Cite

show abstract

“…The output of PCA is reduced low dimensional dataset and the number of dimensions to be selected can be set manually by altering the standard deviation limit. The datasets with different dimensions namely 5, 6,7,8,9,10,13,14,15,18,24, and 42 are taken to compare performance of eight machine learning algorithms to find the minimum dimensions which gives high accuracy rate and low detection time. Figure 4 depicts detection time of eight different machine learning algorithms for different dimensions.…”

Section: Resultsmentioning

confidence: 99%

“…In this work, PCA [10] adopted to reduce the detection time of network based intrusions. KDD99 dataset [7] has been used in this work to measure the detection rate of the following machine learning algorithms Support Vector Machines (SVM), K-Nearest Neighbors (KNN), J48 Tree algorithm, Random Forest Tree classification algorithm, Adaboost algorihm, Nearest Neighbors generalized Exemplars algorithm, Navebayes probabilistic classifier and Voting Features Interval classification algorithms. The rest of the paper is organized as follows.…”

Section: Introductionmentioning

confidence: 99%

Comparative study of Principal Component Analysis based Intrusion Detection approach using machine learning algorithms

Chabathula

C.D

Kumara

2015

2015 3rd International Conference on Signal Processing, Communication and Networking (ICSCN)

View full text Add to dashboard Cite

This paper induces the prominence of variegated machine learning techniques adapted so far for the identifying different network attacks and suggests a preferable Intrusion Detection System (IDS) with the available system resources while optimizing the speed and accuracy. With booming number of intruders and hackers in todays vast and sophisticated computerized world, it is unceasingly challenging to identify unknown attacks in promising time with no false positive and no false negative. Principal Component Analysis (PCA) curtails the amount of data to be compared by reducing their dimensions prior to classification that results in reduction of detection time. In this paper, PCA is adopted to reduce higher dimension dataset to lower dimension dataset. It is accomplished by converting network packet header fields into a vector then PCA applied over high dimensional dataset to reduce the dimension. The reduced dimension dataset is tested with Support Vector Machines (SVM), K-Nearest Neighbors (KNN), J48 Tree algorithm, Random Forest Tree classification algorithm, Adaboost algorihm, Nearest Neighbors generalized Exemplars algorithm, Navebayes probabilistic classifier and Voting Features Interval classification algorithm. Obtained results demonstrates detection accuracy, computational efficiency with minimal false alarms, less system resources utilization. Experimental results are compared with respect to detection rate and detection time and found that TREE classification algorithms achieved superior results over other algorithms. The whole experiment is conducted by using KDD99 data set.

show abstract

“…Reference [9] proposed a hybrid approach combining the information gain ratio (IGR) and the k-means classifier. Reference [10] proposed a feature selection method based on Rough Sets, improved Genetic Algorithms and clustering.…”

Section: Related Workmentioning

confidence: 99%

On Attack-Relevant Ranking of Network Features

Ammar¹,

Al-Shalfan²

2015

ijacsa

View full text Add to dashboard Cite

Abstract-An Intrusion Detection System (IDS) is an important component of the defense-in-depth security mechanism in any computer network system. For assuring timely detection of intrusions from millions of connection records, it is important to reduce the number of connection features examined by the IDS, using feature selection or feature reduction techniques. In this scope, this paper presents the first application of a distinctive feature selection method based on neural networks to the problem of intrusion detection, in order to determine the most relevant network features, which is an important step towards constructing a lightweight anomalybased intrusion detection system. The same procedure is used for feature selection and for attack detection, which gives more consistency to the method. We apply this method to a case study, on KDD dataset and show its advantages compared to some existing feature selection approaches. We then measure its dependence to the network architecture and the learning database.

show abstract

Identifying important characteristics in the KDD99 intrusion detection dataset by feature selection using a hybrid approach

Cited by 33 publications

References 8 publications

Machine-Learning-Based Feature Selection Techniques for Large-Scale Network Intrusion Detection

Machine-Learning-Based Feature Selection Techniques for Large-Scale Network Intrusion Detection

Comparative study of Principal Component Analysis based Intrusion Detection approach using machine learning algorithms

On Attack-Relevant Ranking of Network Features

Contact Info

Product

Resources

About