Clustering Approach Based on Mini Batch Kmeans for Intrusion Detection System Over Big Data

Peng, Kai; Leung, Victor C. M.; Huang, Qilin

doi:10.1109/access.2018.2810267

Cited by 199 publications

(67 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The pros and cons of various shallow models are shown in Table 3. [13] Artificial Neural Network (ANN). The design idea of an ANN is to mimic the way human brains work.…”

Section: Shallow Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey

2019

View full text Add to dashboard Cite

Networks play important roles in modern life, and cyber security has become a vital research area. An intrusion detection system (IDS) which is an important cyber security technique, monitors the state of software and hardware running in the network. Despite decades of development, existing IDSs still face challenges in improving the detection accuracy, reducing the false alarm rate and detecting unknown attacks. To solve the above problems, many researchers have focused on developing IDSs that capitalize on machine learning methods. Machine learning methods can automatically discover the essential differences between normal data and abnormal data with high accuracy. In addition, machine learning methods have strong generalizability, so they are also able to detect unknown attacks. Deep learning is a branch of machine learning, whose performance is remarkable and has become a research hotspot. This survey proposes a taxonomy of IDS that takes data objects as the main dimension to classify and summarize machine learning-based and deep learning-based IDS literature. We believe that this type of taxonomy framework is fit for cyber security researchers. The survey first clarifies the concept and taxonomy of IDSs. Then, the machine learning algorithms frequently used in IDSs, metrics, and benchmark datasets are introduced. Next, combined with the representative literature, we take the proposed taxonomic system as a baseline and explain how to solve key IDS issues with machine learning and deep learning techniques. Finally, challenges and future developments are discussed by reviewing recent representative studies.

show abstract

“…The pros and cons of various shallow models are shown in Table 3. [13] Artificial Neural Network (ANN). The design idea of an ANN is to mimic the way human brains work.…”

Section: Shallow Modelsmentioning

confidence: 99%

“…The standard K-means algorithm is inefficient on big datasets. To improve detection efficiency, Peng et al [13] proposed an improved K-means detection method with mini batch. They first carried out data preprocessing on the KDD99 dataset.…”

Section: Feature Engineering-based Detectionmentioning

confidence: 99%

Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey

2019

View full text Add to dashboard Cite

show abstract

“…PSO was employed to obtain the initial clustering center of the K-means algorithm rather than randomly initializing the K-means algorithm [33]. Compared with the random initialization K-means algorithm, it was more efficient for the PSO-based K-means algorithm to search for the near-global solution or global optimal solution and enhance the clustering accuracy and computational efficiency [34,35].…”

Section: Initial Parameter Calculationmentioning

confidence: 99%

Data-Driven Compartmental Modeling Method for Harmonic Analysis—A Study of the Electric Arc Furnace

Shao

Chen

2019

Energies

View full text Add to dashboard Cite

The electric arc furnace (EAF) contributes to almost one-third of the global iron and steel industry, and its harmonic pollution has drawn attention. An accurate EAF harmonic model is essential to evaluate the harmonic pollution of EAF. In this paper, a data-driven compartmental modeling method (DCMM) is proposed for the multi-mode EAF harmonic model. The proposed DCMM considers the coupling relationship among different frequencies of harmonics to enhance the modeling accuracy, meanwhile, the dimensions of the harmonic dataset are reduced to improve computational efficiency. Furthermore, the proposed DCMM is applicable to establish a multi-mode EAF harmonic model by dividing the multi-mode EAF harmonic dataset into several clusters corresponding to the different modes of the EAF smelting process. The performance evaluation results show that the proposed DCMM is adaptive in terms of establishing the multi-mode model, even if the data volumes, number of clusters, and sample distribution change significantly. Finally, a case study of EAF harmonic data is conducted to establish a multi-mode EAF harmonic model, showing that the proposed DCMM is effective and accurate in EAF modeling.

show abstract

“…J. Li and H. Liu et al [18] have envisioned the challenges for big data analytics. To facilitate and promote feature selection research, they present an open source feature selection repository (scikitfeature) of popular algorithms.…”

Section: Related Workmentioning

confidence: 99%

KM-MBFO: A Hybrid Hadoop Map Reduce Access for Clustering Big Data by Adopting Modified Bacterial Foraging Optimization Algorithm

2019

IJRTE

View full text Add to dashboard Cite

K-Means Clustering is a very powerful and frequently used algorithm for the clustering, it has got its own limitation. The prevalent K-Means clustering algorithm used for grouping have inadequacies, for example, slow convergence rate, local optima trap, and so on. Therefore, many swarm knowledge based procedures combined with KM for clustering were presented and demonstrated their presentation, its variations and its applications in data grouping. In this paper we intend to propose a parallel organizing strategy for KM-MBFO mechanism that actualized in Hadoop Distributed File System (HDFS) for diminishing the execution time. This Mapper approach produces the populace for given data set for grouping. The Modified Bacterial Foraging Optimization (MBFO) algorithm finds the wellness of the populace to choose the optimal K values as far as execution time and classification error. Through simulated test results, we assess the demonstration of the proposed KM-BFO conspire

show abstract

Clustering Approach Based on Mini Batch Kmeans for Intrusion Detection System Over Big Data

Cited by 199 publications

References 25 publications

Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey

Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey

Data-Driven Compartmental Modeling Method for Harmonic Analysis—A Study of the Electric Arc Furnace

KM-MBFO: A Hybrid Hadoop Map Reduce Access for Clustering Big Data by Adopting Modified Bacterial Foraging Optimization Algorithm

Contact Info

Product

Resources

About