Abstracting massive data for lightweight intrusion detection in computer networks

Wang, Wei; Liu, Jiqiang; Pitsilis, Georgios; Zhang, Xiangliang

doi:10.1016/j.ins.2016.10.023

Cited by 63 publications

(35 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A case study using data from the UCI Machine Learning Repository [97] reduced an initial set of 4764 features to 623 [98]. For instance reduction, Wang et al (2016) employ a framework based on two clustering algorithms, affinity propagation (AP) and k-nearest neighbor (k-NN), to extract "exemplars", or representations of some number of actual data points [99]. A clustering algorithm is employed to cluster the data instances into similar groups; an exemplar is then defined to represent the group.…”

Section: Framework For Data Reductionmentioning

confidence: 99%

“…Even if TSFRESH then filters out 50% of the features, there still could remain many hundreds of features in the model. This could be an excessive number of features that strains the capacity of the analyst to truly grasp what is going on For instance reduction, Wang et al (2016) employ a framework based on two clustering algorithms, affinity propagation (AP) and k-nearest neighbor (k-NN), to extract "exemplars", or representations of some number of actual data points [99]. A clustering algorithm is employed to cluster the data instances into similar groups; an exemplar is then defined to represent the group.…”

Section: Framework For Data Reductionmentioning

confidence: 99%

See 1 more Smart Citation

A Survey of Feature Set Reduction Approaches for Predictive Analytics Models in the Connected Manufacturing Enterprise

2019

View full text Add to dashboard Cite

The broad context of this literature review is the connected manufacturing enterprise, characterized by a data environment such that the size, structure and variety of information strain the capability of traditional software and database tools to effectively capture, store, manage and analyze it. This paper surveys and discusses representative examples of existing research into approaches for feature set reduction in the big data environment, focusing on three contexts: general industrial applications; specific industrial applications such as fault detection or fault prediction; and data reduction. The conclusion from this review is that there is room for research into frameworks or approaches to feature filtration and prioritization, specifically with respect to providing quantitative or qualitative information about the individual features in the dataset that can be used to rank features against each other. A byproduct of this gap is a tendency for analysts not to holistically generalize results beyond the specific problem of interest, and, related, for manufacturers to possess only limited knowledge of the relative value of smart manufacturing data collected.

show abstract

Section: Framework For Data Reductionmentioning

confidence: 99%

Section: Framework For Data Reductionmentioning

confidence: 99%

A Survey of Feature Set Reduction Approaches for Predictive Analytics Models in the Connected Manufacturing Enterprise

2019

View full text Add to dashboard Cite

show abstract

“…where ( ) is the information entropy of a sample and ( | ) is the conditional entropy of feature in the sample, which represents the information quantity (i.e., exists or does not exist in the classification system) [26,27].…”

Section: Information Gain Information Gain (Ig)mentioning

confidence: 99%

Computing Adaptive Feature Weights with PSO to Improve Android Malware Detection

Zheng

et al. 2017

Security and Communication Networks

View full text Add to dashboard Cite

Android malware detection is a complex and crucial issue. In this paper, we propose a malware detection model using a support vector machine (SVM) method based on feature weights that are computed by information gain (IG) and particle swarm optimization (PSO) algorithms. The IG weights are evaluated based on the relevance between features and class labels, and the PSO weights are adaptively calculated to result in the best fitness (the performance of the SVM classification model). Moreover, to overcome the defects of basic PSO, we propose a new adaptive inertia weight method called fitness-based and chaotic adaptive inertia weight-PSO (FCAIW-PSO) that improves on basic PSO and is based on the fitness and a chaotic term. The goal is to assign suitable weights to the features to ensure the best Android malware detection performance. The results of experiments indicate that the IG weights and PSO weights both improve the performance of SVM and that the performance of the PSO weights is better than that of the IG weights.

show abstract

“…In order to detect anomalies in network, correlate parameters from different layers should be combined [8]. Some papers focus on building a new hierarchical framework for intrusion detection as well as data processing based on the feature classification and selection [9][10][11].…”

Section: Anomaly Detectionmentioning

confidence: 99%

A Universal High-Performance Correlation Analysis Detection Model and Algorithm for Network Intrusion Detection System

Zhu

Liu

Sun

et al. 2017

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In big data era, the single detection techniques have already not met the demand of complex network attacks and advanced persistent threats, but there is no uniform standard to make different correlation analysis detection be performed efficiently and accurately. In this paper, we put forward a universal correlation analysis detection model and algorithm by introducing state transition diagram. Based on analyzing and comparing the current correlation detection modes, we formalize the correlation patterns and propose a framework according to data packet timing and behavior qualities and then design a new universal algorithm to implement the method. Finally, experiment, which sets up a lightweight intrusion detection system using KDD1999 dataset, shows that the correlation detection model and algorithm can improve the performance and guarantee high detection rates.

show abstract

Abstracting massive data for lightweight intrusion detection in computer networks

Cited by 63 publications

References 22 publications

A Survey of Feature Set Reduction Approaches for Predictive Analytics Models in the Connected Manufacturing Enterprise

A Survey of Feature Set Reduction Approaches for Predictive Analytics Models in the Connected Manufacturing Enterprise

Computing Adaptive Feature Weights with PSO to Improve Android Malware Detection

A Universal High-Performance Correlation Analysis Detection Model and Algorithm for Network Intrusion Detection System

Contact Info

Product

Resources

About