A survey of deep learning-based network anomaly detection

Kwon, Donghwoon; Kim, Hyun‐Joo; Kim, Jin Oh; Suh, Sang C.; Kim, Ikkyun; Kim, Kuinam J.

doi:10.1007/s10586-017-1117-8

Cited by 500 publications

(275 citation statements)

References 25 publications

Supporting

Mentioning

267

Contrasting

Unclassified

Order By: Relevance

“…Researchers such as Kolosnjaji et al uses convolutional and recurrent network to obtain the best feature set. AE provides a good solution to the clustering problem . The autoencoder comprised of two portions, ie, the encoder and the decoder section.…”

Section: Related Workmentioning

confidence: 99%

“…AE provides a good solution to the clustering problem. 14 The autoencoder comprised of two portions, ie, the encoder and the decoder section. Autoencoder reduces the concern of the large feature space.…”

Section: Related Workmentioning

confidence: 99%

“…To date, there are considerable work via Machine Learning (ML) to resolve the malware clustering issue, but very few work has been proposed by use of deep learning as a tool for malware clustering. In the literature, the potential adoption of deep learning has shifted from the vision and speech to malware detection.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Static malware clustering using enhanced deep embedding method

Jiang

Zhang

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

Summary Malware refers to any software, programs, or files that are intentionally utilised to compromise the system and cause unexpected losses to end‐users such as economical losses or privacy breaches. The rapid growth of malware makes it impossible to keep up with its progress merely via human interventions or manual analysis. One of the challenges for the human‐oriented approaches is they will cause backlog and inability to keep up with the development traces of the malware. Hence, an efficient method is needed urgently to analyse effectively and identify accurately the malware in their domain. Malware clustering has been extensively studied in the machine learning area with regards to distance functions, grouping algorithm and cluster validation. A large number of research studies have been done via behavioral analysis for clustering to achieve high performance of malware detections. However, there is a trade‐off for better detection performance between behaviorial approaches and high computational forces. Up to date, little work focuses on the deep learning representations for malware clustering. Therefore, in this paper, we propose an enhanced deep embedded clustering method to facilitate an effective and efficient malware clustering process. The new method takes advantage of linear dimensionality reduction and a customised deep neural network to learn malware representations in an orthogonal space and performs cluster assignments. Our experimental results demonstrate that the proposed clustering model outperforms the traditional K‐means method with regards to the enhanced features using various auto‐encoder, pre‐trained weight and principle component analysis (PCA).

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Static malware clustering using enhanced deep embedding method

Jiang

Zhang

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…The increasing incorporation of Cyber-based methodologies for monitoring and control of physical systems has made critical infrastructure vulnerable to various cyber-attacks such as: interception, removal or replacement of information, penetration of unauthorized users and viruses [1] [2] [3] [4]. Intrusion detection systems (IDSs) are an essential tool to detect such malicious attacks [5].…”

Section: Introductionmentioning

confidence: 99%

An Adversarial Approach for Explainable AI in Intrusion Detection Systems

Marino

Wickramasinghe

Manic

2018

IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society

View full text Add to dashboard Cite

Despite the growing popularity of modern machine learning techniques (e.g. Deep Neural Networks) in cyber-security applications, most of these models are perceived as a black-box for the user. Adversarial machine learning offers an approach to increase our understanding of these models. In this paper we present an approach to generate explanations for incorrect classifications made by data-driven Intrusion Detection Systems (IDSs) An adversarial approach is used to find the minimum modifications (of the input features) required to correctly classify a given set of misclassified samples. The magnitude of such modifications is used to visualize the most relevant features that explain the reason for the misclassification. The presented methodology generated satisfactory explanations that describe the reasoning behind the mis-classifications, with descriptions that match expert knowledge. The advantages of the presented methodology are: 1) applicable to any classifier with defined gradients. 2) does not require any modification of the classifier model. 3) can be extended to perform further diagnosis (e.g. vulnerability assessment) and gain further understanding of the system. Experimental evaluation was conducted on the NSL-KDD99 benchmark dataset using Linear and Multilayer perceptron classifiers. The results are shown using intuitive visualizations in order to improve the interpretability of the results.

show abstract

“…So far, we have seen enormous work on automated diagnostics of IT systems, with use of data mining or Artificial Intelligence (AI) [2], [3]. Most of this work uses for diagnostics either metrics or logs.…”

Section: Introductionmentioning

confidence: 99%

Next Stop "NoOps": Enabling Cross-System Diagnostics Through Graph-Based Composition of Logs and Metrics

Zasadzinski¹,

Solé²,

Brandon

et al. 2018

2018 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

Performing diagnostics in IT systems is an increasingly complicated task, and it is not doable in satisfactory time by even the most skillful operators. Systems and their architecture change very rapidly in response to business and user demand. Many organizations see value in the maintenance and management model of NoOps that stands for No Operations. One of the implementations of this model is a system that is maintained automatically without any human intervention. The path to NoOps involves not only precise and fast diagnostics but also reusing as much knowledge as possible after the system is reconfigured or changed. The biggest challenge is to leverage knowledge on one IT system and reuse this knowledge for diagnostics of another, different system. We propose a framework of weighted graphs which can transfer knowledge, and perform high-quality diagnostics of IT systems. We encode all possible data in a graph representation of a system state and automatically calculate weights of these graphs. Then, thanks to the evaluation of similarity between graphs, we transfer knowledge about failures from one system to another and use it for diagnostics. We successfully evaluate the proposed approach on Spark, Hadoop, Kafka and Cassandra systems.

show abstract

A survey of deep learning-based network anomaly detection

Cited by 500 publications

References 25 publications

Static malware clustering using enhanced deep embedding method

Static malware clustering using enhanced deep embedding method

An Adversarial Approach for Explainable AI in Intrusion Detection Systems

Next Stop "NoOps": Enabling Cross-System Diagnostics Through Graph-Based Composition of Logs and Metrics

Contact Info

Product

Resources

About