Building Auto-Encoder Intrusion Detection System based on random forest feature selection

Li, XuKui; Chen, Wei; Zhang, Qianru; Wu, Lifa

doi:10.1016/j.cose.2020.101851

Cited by 225 publications

(111 citation statements)

References 17 publications

Supporting

Mentioning

102

Contrasting

Unclassified

Order By: Relevance

“…These noticeably high scores for the various metrics may be due to overfitting. Surprisingly, use of the accuracy metric is prevalent throughout the surveyed works, while use of the AUC metric has only been used in four studies (Fitni & Ramli, 2020;Kanimozhi & Jacob, 2019a;Kanimozhi & Jacob, 2019b;Li et al, 2020). This observation relates to the class imbalance of CICIDS2018.…”

Section: Discussion Of Surveyed Workmentioning

confidence: 99%

A survey and analysis of intrusion detection models based on CSE-CIC-IDS2018 Big Data

2020

View full text Add to dashboard Cite

The exponential growth in computer networks and network applications worldwide has been matched by a surge in cyberattacks. For this reason, datasets such as CSE-CIC-IDS2018 were created to train predictive models on network-based intrusion detection. These datasets are not meant to serve as repositories for signature-based detection systems, but rather to promote research on anomaly-based detection through various machine learning approaches. CSE-CIC-IDS2018 contains about 16,000,000 instances collected over the course of ten days. It is the most recent intrusion detection dataset that is big data, publicly available, and covers a wide range of attack types. This multi-class dataset has a class imbalance, with roughly 17% of the instances comprising attack (anomalous) traffic. Our survey work contributes several key findings. We determined that the best performance scores for each study, where available, were unexpectedly high overall, which may be due to overfitting. We also found that most of the works did not address class imbalance, the effects of which can bias results in a big data study. Lastly, we discovered that information on the data cleaning of CSE-CIC-IDS2018 was inadequate across the board, a finding that may indicate problems with reproducibility of experiments. In our survey, major research gaps have also been identified.

show abstract

Section: Discussion Of Surveyed Workmentioning

confidence: 99%

A survey and analysis of intrusion detection models based on CSE-CIC-IDS2018 Big Data

2020

View full text Add to dashboard Cite

show abstract

“…The memory size of the AE will also be constant as the shape of the weight matrices will be unchanged throughout the data stream, which is an advantage over a decision tree method such as HAT, that can grow with the stream [3]. This paper has focused solely on the autoencoder, whereas other studies [9,11,14] have augmented the model with various pre and post processing stages, that could be unnecessary when considering the raw performance of the AE itself.…”

Section: Discussionmentioning

confidence: 99%

“…Li et al [9] expanded on the Kitsune online ensemble method [11], using random forest for feature selection and GMM/K-Means to normalise the output RE which is calculated based on the Root Mean Squared Error (RMSE). The claim is that using random forest for feature selection is more efficient than the clustering approach used with Kitsune.…”

Section: Related Workmentioning

confidence: 99%

Autoencoders

Nixon

Sedky

Hassan

2020

Proceedings of the 2020 4th International Conference on Cloud and Big Data Computing

View full text Add to dashboard Cite

Computer networks are vulnerable to cyber attacks that can affect the confidentiality, integrity and availability of mission critical data. Intrusion detection methods can be employed to detect these attacks in real-time. Anomaly detection offers the advantage of detecting unknown attacks in a semi-supervised fashion. This paper aims to answer the question if autoencoders, a type of semi-supervised feedforward neural network, can provide a low cost anomaly detector method for computer network data streams. Autoencoder methods were evaluated online with well known KDD Cup 1999 and UNSW-NB15 data sets, and it was demonstrated that running time and labeling cost is significantly reduced compared to traditional online classification techniques for similar detection performance. Further research would consider the trade-off between single vs stacked networks, multi-label classification, concept drift detection and active learning.

show abstract

“…Performing this for all trees and averaging for each variable, it is possible to have a relevance comparison metric, which is usually referred to as Variable Importance Measure (VIM) or Permutation Importance Index (PIM). Another way to create an importance rank is, on each tree and each node split, to calculate the split improvement by a measure (eg, Gini Index) and use these values to compare the variables' importance 9 …”

Section: Introductionmentioning

confidence: 99%

“…In studying the behavior of social animals with the artificial life theory, for how to construct the swarm artificial life systems with cooperative behavior by computer, Millonas proposed five basic principles 9 Proximity: the swarm should be able to carry out simple space and time computations. Quality: the swarm should be able to sense the quality change in the environment and response it. Diverse response: the swarm should not limit its way to get the resources in a narrow scope. Stability: the swarm should not change its behavior mode with every environmental change. Adaptability: the swarm should change its behavior mode when this change is worthy. …”

Section: Introductionmentioning

confidence: 99%

Introduced a new method for enhancement of intrusion detection with random forest and PSO algorithm

Ajdani

Ghaffary

2021

Security and Privacy

View full text Add to dashboard Cite

As computer networks expand, attacks and intrusions into these networks have increased. In addition to firewalls and other intrusion prevention equipment, other systems, such as IDS (Metrics), are designed to provide enhanced security in computer systems, including the purpose of monitoring intrusive and intrusive activities. Intrusive allocation system can be considered effective if the high intrusion rate is slightly misleading, and in this article a new way to classify it is abnormal (infiltration) in the host or network. UNSW‐NB15 and KDD‐Cup'99 Datasets Introducing Random Forest and PSO algorithm. In this paper, training data and label data used with the random forest algorithm. After creating a random forest algorithm, provide test data. We use the data stored in train step, which is actually a copy of the data, so that when performing the test step, the same training data can be compared with categorize using a PSO algorithm. In in order to show the accuracy of the proposed method, an example of the confusion matrix formed in the code that showed performance of all methods and modes studied is compared based on accuracy and time that the PSO algorithm has always been able to take less time, which is quite acceptable and predictable. Improves correct diagnosis of correct detection rate in that report was 75.94% and in the proposed method in this article it reached 97%. With the proposed method, learning speed has been greatly increased and accuracy is acceptable.

show abstract

Building Auto-Encoder Intrusion Detection System based on random forest feature selection

Cited by 225 publications

References 17 publications

A survey and analysis of intrusion detection models based on CSE-CIC-IDS2018 Big Data

A survey and analysis of intrusion detection models based on CSE-CIC-IDS2018 Big Data

Autoencoders

Introduced a new method for enhancement of intrusion detection with random forest and PSO algorithm

Contact Info

Product

Resources

About