Scalable Hypergrid k-NN-Based Online Anomaly Detection in Wireless Sensor Networks

Xie, Miao; Hu, Jiankun; Han, Sunyoung; Chen, Hsiao‐Hwa

doi:10.1109/tpds.2012.261

Cited by 117 publications

(66 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, due to a large number of computations required to find the distance from each new sample to all of the training samples, KNN is time consuming. Higher level structures like hyper-grids are applied to solve the problem and make it faster, but it is still not fast enough for online applications [18]. The distance that is used in a KNN-based method depends on the application.…”

Section: Proximity-based Methodsmentioning

confidence: 99%

A fast and noise resilient cluster-based anomaly detection

Bigdeli

Mohammadi

Raahemi

et al. 2015

Pattern Anal Applic

View full text Add to dashboard Cite

Clustering, while systematically applied in anomaly detection, has a direct impact on the accuracy of the detection methods. Existing cluster-based anomaly detection methods are mainly based on spherical shape clustering. In this paper, we focus on arbitrary shape clustering methods to increase the accuracy of the anomaly detection. However, since the main drawback of arbitrary shape clustering is its high memory complexity, we propose to summarize clusters first. For this, we design an algorithm, called Summarization based on Gaussian Mixture Model (SGMM), to summarize clusters and represent them as Gaussian Mixture Models (GMMs). After GMMs are constructed, incoming new samples are presented to the GMMs, and their membership values are calculated, based on which the new samples are labeled as ''normal'' or ''anomaly.'' Additionally, to address the issue of noise in the data, instead of labeling samples individually, they are clustered first, and then each cluster is labeled collectively. For this, we present a new approach, called Collective Probabilistic Anomaly Detection (CPAD), in which, the distance of the incoming new samples and the existing SGMMs is calculated, and then the new cluster is labeled the same as of the closest cluster. To measure the distance of two GMM-based clusters, we propose a modified version of the Kullback-Libner measure. We run several experiments to evaluate the performances of the proposed SGMM and CPAD methods and compare them against some of the wellknown algorithms including ABACUS, local outlier factor (LOF), and one-class support vector machine (SVM). The performance of SGMM is compared with ABACUS using Dunn and DB metrics, and the results indicate that the SGMM performs superior in terms of summarizing clusters. Moreover, the proposed CPAD method is compared with the LOF and one-class SVM considering the performance criteria of (a) false alarm rate, (b) detection rate, and (c) memory efficiency. The experimental results show that the CPAD method is noise resilient, memory efficient, and its accuracy is higher than the other methods.

show abstract

Section: Proximity-based Methodsmentioning

confidence: 99%

A fast and noise resilient cluster-based anomaly detection

Bigdeli

Mohammadi

Raahemi

et al. 2015

Pattern Anal Applic

View full text Add to dashboard Cite

show abstract

“…Higher-level structures like hyper-grid have been tried to address this problem, and make it faster, but, it is still not fast enough for online applications [Xie et al, 2013].…”

Section: K-nearest-neighbors (Knn)mentioning

confidence: 99%

Incremental anomaly detection using two-layer cluster-based structure

Bigdeli

Mohammadi

Raahemi

et al. 2018

Information Sciences

View full text Add to dashboard Cite

Anomaly detection algorithms face several challenges, including processing speed and dealing with noise in data. In this thesis, a two-layer clusterbased anomaly detection structure is presented which is fast, noise-resilient and incremental. In this structure, each normal pattern is considered as a cluster, and each cluster is represented using a Gaussian Mixture Model (GMM). Then, new instances are presented to the GMM to be labeled as normal or abnormal.The proposed structure comprises three main steps. In the first step, the data are clustered. The second step is to represent each cluster in a way that enables the model to classify new instances. The Summarization based on Gaussian Mixture Model (SGMM) proposed in this thesis represents each cluster as a GMM.In the third step, a two-layer structure efficiently updates clusters using In most real-time anomaly detection applications, incoming instances are often similar to previous ones. In these cases, there is no need to update clusters based on duplicates, since they have already been modeled in the cluster distribution. The two-layer structure is responsible for identifying redundant instances. In this structure, redundant instance are ignored, and the remaining new instances are used to update clusters. Ignoring redundant instances, which are typically in the majority, makes the detection phase fast.Each part of the general structure is validated in this thesis. The experiments include, detection rates, clustering goodness, time, memory usage and the complexity of the algorithms. The accuracy of the clustering and summarization of clusters using GMMs is evaluated, and compared to that of other methods. Using Davies-Bouldin (DB) and Dunn indexes, the distances for original and regenerated clusters using GMMs is almost zero with SGMM method while this value for ABACUS is around 0.01. Moreover, the results show that the SGMM algorithm is 3 times faster than ABACUS in running time, using one-third of the memory used by ABACUS.The CPL method, used to label new instances, is found to collectively remove the effect of noise, while increasing the accuracy of labeling new instances. In a noisy environment, the detection rate of the CPL method is 5% higher than other algorithms such as one-class SVM. The false alarm iii rate is decreased by 10% on average. Memory use is 20 times lesser that that of the one-class SVM.The proposed method is found to lower the false alarm rate, which is one of the basic problems for the one-class SVM. Experiments show the false alarm rate is decreased from 5% to 15% among different datasets, while the detection rate is increased from 5% to 10% in different datasets with twolayer structure. The memory usage for the two-layer structure is 20 to 50 times less than that of one-class SVM. One-class SVM uses support vectors in labeling new instances, while the labeling of the two-layer structure depends on the number of GMMs. The experiments show that the two-layer structure is 20 to 50 times faster than the one-class SVM in labelin...

show abstract

“…The F-measure of their scheme can be increased, if the number of classifiers can be added and tested. Xie et al [13] then proposed a KNN-classification based anomaly detection technique in which hyper grid intuition based approach is applied. The computational complexity is reduced by redefining anomalies from hypersphere detection region to hypercube detection region.…”

Section: Related Workmentioning

confidence: 99%

An Efficient Hybrid Anomaly Detection Scheme Using K-Means Clustering for Wireless Sensor Networks

Wazid

Das

2016

Wireless Pers Commun

View full text Add to dashboard Cite

Sensor nodes in a wireless sensor network (WSN) may be lost due to enervation or malicious attacks by an adversary. WSNs deployed for several applications including military applications are prone to various attacks, which degrade the network performance very rapidly. Hybrid anomaly is a type of anomaly that contains the different types of attacker nodes such as blackhole, misdirection, wormhole etc. These multiple attacks can be launched in the network using the hybrid anomaly. In this situation, it is very difficult to find out which kind of attacker nodes are activated in the network. This motivates us to design a robust and efficient secure intrusion detection approach in order to extend the lifetime of a WSN. In this paper, we aim to propose a new intrusion detection technique for hybrid anomaly, which uses the existing data mining algorithm, called K-means clustering. For the detection purpose, patterns of intrusions are built automatically by the K-means clustering algorithm over training data. After that intrusions are detected by matching network activities against these detection patterns. We evaluate our approach over a WSN dataset that is created using Opnet modeler, which contains various attributes, such as endto-end delay, traffic sent and traffic received. The training dataset contains the normal values of the network parameters. The testing dataset is created in actual working mode consists of normal and abnormal values of the network parameters. The proposed technique has the ability to detect two types of malicious nodes: blackhole and misdirection nodes. Our scheme achieves 98.6 % detection rate and 1.2 % false positive rate, which are significantly better than the existing related schemes.

show abstract

Scalable Hypergrid k-NN-Based Online Anomaly Detection in Wireless Sensor Networks

Cited by 117 publications

References 24 publications

A fast and noise resilient cluster-based anomaly detection

A fast and noise resilient cluster-based anomaly detection

Incremental anomaly detection using two-layer cluster-based structure

An Efficient Hybrid Anomaly Detection Scheme Using K-Means Clustering for Wireless Sensor Networks

Contact Info

Product

Resources

About