CLAPP: A self constructing feature clustering approach for anomaly detection

Kumar, Gunupudi Rajesh; Mangathayaru, Nimmala; Narsimha, G.; Reddy, Gali Suresh

doi:10.1016/j.future.2016.12.040

Cited by 68 publications

(24 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zolanvari et al (2019) demonstrates their model by deploying backdoor, command injection and structured query language injection attacks and proved their machine learning model performed well against anomalies. In Gunupudi, Nimmala, Gugulothu, and Gali (2017), a self‐constructing feature clustering technique that applies a Gaussian membership function to anomaly detection in IoT network is proposed. A Gaussian dissimilarity measure Aljawarneh and Vangipuram (2018) is proposed to perform similarity computation for anomaly detection in IoT environment.…”

Section: Literature Surveymentioning

confidence: 99%

“…Nguyen et al (2019) proposed a new collaborative and intelligent NIDS architecture for anomaly detection in SDN‐based cloud IoT networks which obtained better detection accuracy results for DDoS attacks. Gunupudi et al (2017) Kumar, Mangathayaru, Narsimha, and Cheruvu (2018) Mangathayaru, Kumar, and Narsimha (2016) Kumar, Mangathayaru, Narsimha, and Reddy (2017) propose self‐constructing feature clustering technique for anomaly detection using Gaussian‐based approach. Li et al (2019) demonstrates system statistics learning‐based IoT security and introduced a neural network classifier algorithm for anomaly detection.…”

Section: Literature Surveymentioning

confidence: 99%

See 1 more Smart Citation

A machine learning approach for imputation and anomaly detection in IoT environment

Radhakrishna

Kumar

Puligadda³

et al. 2020

Expert Systems

Self Cite

View full text Add to dashboard Cite

The problem of anomaly and attack detection in IoT environment is one of the prime challenges in the domain of internet of things that requires an immediate concern. For example, anomalies and attacks in IoT environment such as scan, malicious operation, denial of service, spying, data type probing, wrong setup, malicious control can lead to failure of an IoT system. Datasets generated in an IoT environment usually have missing values. The presence of missing values makes the classifier unsuitable for classification task. This article introduces (a) a novel imputation technique for imputation of missing data values (b) a classifier which is based on feature transformation to perform classification (c) imputation measure for similarity computation between any two instances that can also be used as similarity measure. The performance of proposed classifier is studied by using imputed datasets obtained through applying Kmeans, F-Kmeans and proposed imputation methods. Experiments are also conducted by applying existing and proposed classifiers on the imputed dataset obtained using proposed imputation technique. For experimental study in this article, we have used an open source dataset named distributed smart space orchestration system publicly available from Kaggle. Experiment results obtained are also validated using Wilcoxon non-parametric statistical test. It is proved that the performance of proposed approach is better when compared to existing classifiers when the imputation process is performed using F-Kmeans and K-Means imputation techniques. It is also observed that accuracies for attack classes scan, malicious operation, denial of service, spying, data type probing, wrong setup are 100% while it is 99% for malicious control attack class when the proposed imputation and classification technique are applied.

show abstract

Section: Literature Surveymentioning

confidence: 99%

Section: Literature Surveymentioning

confidence: 99%

A machine learning approach for imputation and anomaly detection in IoT environment

Radhakrishna

Kumar

Puligadda³

et al. 2020

Expert Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is important to detect the outliers efficiently and accurately to improve the reliability of WSN data. The general outlier detection methods can be classified into four classes: statistical-based methods, [4][5][6] nearest neighbor-based methods, 7,9 clustering-based methods, [10][11][12] and classification-based methods. [13][14][15][16][17] Statistical-based methods capture the distribution of the data and evaluate how well the data instance matches the model.…”

Section: Related Workmentioning

confidence: 99%

“…Finally, collected datasets have high dimensionality and large scalability for certain cases, presenting issues for data processing. In the past several years, numerous methods have been proposed to perform outlier detection for WSNs [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19] (reviewed in section ''Related work''). However, the majority of these can only address the first two challenges, and most of them cannot be directly applied to high-dimensional and large-scalability data because of the following issues: 15 (1) time-consuming-as the dimension of the input data vector increases, the number of feature subspaces increases exponentially, which results in an exponential search space; (2) low detection rate-the high proportion of irrelevant features in highdimensional datasets unavoidably include noises, which makes the true outliers inconspicuous; and (3) high false alarm rate-in high-dimensional space, we can always determine at least one feature subspace for each point of a dataset that defines such a point as an outlier, that is, every data instance can be considered as an outlier under a particular circumstance.…”

Section: Introductionmentioning

confidence: 99%

Fast outlier detection for high-dimensional data of wireless sensor networks

Qiao

Cui

Jin

et al. 2020

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

This article addresses the problem of outlier detection for wireless sensor networks. As increasing amounts of observational data are tending to be high-dimensional and large scale, it is becoming increasingly difficult for existing techniques to perform outlier detection accurately and efficiently. Although dimensionality reduction tools (such as deep belief network) have been utilized to compress the high-dimensional data to support outlier detection, these methods may not achieve the desired performance due to the special distribution of the compressed data. Furthermore, because most existed classification methods must solve a quadratic optimization problem in their training stage, they cannot perform well in large-scale datasets. In this article, we developed a new form of classification model called “deep belief network online quarter-sphere support vector machine,” which combines deep belief network with online quarter-sphere one-class support vector machine. Based on this model, we first propose a model training method that learns the radius of the quarter sphere by a sorting method. Then, an online testing method is proposed to perform online outlier detection without supervision. Finally, we compare the proposed method with the state of the arts using extensive experiments. The experimental results show that our method not only reduces the computational cost by three orders of magnitude but also improves the detection accuracy by 3%–5%.

show abstract

“…Feature selection [1], Feature representation [19], [20] and dimensionality reduction approaches [21]- [23] have been studied and extensively addressed in many research contributions related to text classification, data fusion, image fusion, medical data classification and various machine learning and data mining applications. Feature reduction techniques are also applied for the design of intrusion detection systems (IDS) [19], [20] in the literature. Several studies are also carried on how to choose a right classifier and apply it for building efficient network intrusion detection [1].…”

Section: Introductionmentioning

confidence: 99%

Similarity Based Feature Transformation for Network Anomaly Detection

Nagaraja

Boregowda²,

Khatatneh

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The fundamental objective behind any network intrusion detection system is to automate the detection process whenever intrusions occur in the network. The problem of the network anomaly detection is to determine, if the network incoming traffic is legitimate (or) anomalous. Automated detection systems designed to identify incoming anomalous traffic patterns usually apply widely used machine learning techniques. However, irrespective of any system model which is developed to identify anomalous traffic, all these models requires comparing anomalous and normal traffic patterns. Such comparisons implicitly depend on the ability of the underlying machine learning model to gauge the similarity between a known legitimate observation and the target. The efficiency of any network anomalous detection system depends on the use of distance (or) similarity measures and how they are actually applied. A novel distance function which can be applied to determine the similarity between two conditional feature pattern vectors is an important contribution of present research. Feature dimensionality is another important issue for any machine learning algorithm. In the present work, feature reduction is achieved using the proposed feature transformation technique. However, our approach for feature transformation uses the proposed gaussian distance function to achieve dimensionality reduction to represent the original input dataset in the new transformation space. We have also proposed new computation expressions for determining equivalent deviation and threshold in gaussian space. Experiments are performed on KDD and NSL-KDD datasets by considering widely applied classifier algorithms in various state-of-art research contributions. For performance validation of machine learning models, k-fold cross validation is applied by setting k to 10 through considering evaluation parameters such as accuracy, precision and recall. Experiment results have proved that our approach for anomaly detection that applies the proposed feature transformation technique proved comparatively better to detection methods CANN, GARUDA, and UTTAMA addressed in the recent research literature.

show abstract

CLAPP: A self constructing feature clustering approach for anomaly detection

Cited by 68 publications

References 23 publications

A machine learning approach for imputation and anomaly detection in IoT environment

A machine learning approach for imputation and anomaly detection in IoT environment

Fast outlier detection for high-dimensional data of wireless sensor networks

Similarity Based Feature Transformation for Network Anomaly Detection

Contact Info

Product

Resources

About