An Anomaly Detection Based on Optimization

Alguliyev, Rasim; Alıguliyev, Ramiz M.; İmamverdiyev, Yadigar; Sukhostat, Lyudmila

doi:10.5815/ijisa.2017.12.08

Cited by 19 publications

(6 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A large number of researches have been devoted to big data analysis [3, 4, 6, 7]. The issue with big data clustering is that a lot of memory is required.…”

Section: Related Workmentioning

confidence: 99%

Efficient algorithm for big data clustering on single machine

Alguliyev

Alıguliyev

Sukhostat

2020

CAAI trans. intell. technol.

Self Cite

View full text Add to dashboard Cite

Big data analysis requires the presence of large computing powers, which is not always feasible. And so, it became necessary to develop new clustering algorithms capable of such data processing. This study proposes a new parallel clustering algorithm based on the k‐means algorithm. It significantly reduces the exponential growth of computations. The proposed algorithm splits a dataset into batches while preserving the characteristics of the initial dataset and increasing the clustering speed. The idea is to define cluster centroids, which are also clustered, for each batch. According to the obtained centroids, the data points belong to the cluster with the nearest centroid. Real large datasets are used to conduct the experiments to evaluate the effectiveness of the proposed approach. The proposed approach is compared with k‐means and its modification. The experiments show that the proposed algorithm is a promising tool for clustering large datasets in comparison with the k‐means algorithm.

show abstract

“…A large number of researches have been devoted to big data analysis [3, 4, 6, 7]. The issue with big data clustering is that a lot of memory is required.…”

Section: Related Workmentioning

confidence: 99%

Efficient algorithm for big data clustering on single machine

Alguliyev

Alıguliyev

Sukhostat

2020

CAAI trans. intell. technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Big data analytics is the process of detecting hidden didactics, unknown correlation and other useful information in large volumes of data for making optimal (the best) decision. In this context, anomaly detection is in the focus of attention both in scientific research and application fields as a very serious problem [1,2,[9][10][11].…”

Section: Review Of Computer Engineering Researchmentioning

confidence: 99%

“…In numerous studies, statistical approaches for detection of anomalies reduction of size, based on machine learning, neural networks, the Bayesian network, entropy, based on rules and optimization, SVM-based, etc. models and algorithms were proposed [8,10,16,17].…”

confidence: 99%

“…In many studies, hybrid or multi-level classification models have been suggested to increase the accuracy of classification in the detection of anomaly [8,10,[19][20][21][22]. The combined use of numerous data mining methods is known as an ensemble approach, and the process of learning the correlation between these ensemble techniques is known by names such as meta-learning.…”

confidence: 99%

See 1 more Smart Citation

Classification Ensemble Based Anomaly Detection in Network Traffic

Alıguliyev

Hajirahimova

2019

Review of Computer Engineering Research

View full text Add to dashboard Cite

Recently, the expansion of information technologies and the exponential increase of the digital data have deepened more the security and confidentiality issues in computer networks. In the Big Data era information security has become the main direction of scientific research and Big Data analytics is considered being the main tool in the solution of information security issue. Anomaly detection is one of the main issues in data analysis and used widely for detecting network threats. The potential sources of outliers can be noise and errors, events, and malicious attacks on the network. In this work, a short review of network anomaly detection methods is given, is looked at related works. In the article, a more exact and simple multi-classifier model is proposed for anomaly detection in network traffic based on Big Data. Experiments have been performed on the NSL-KDD data set by using the Weka. The offered model has shown decent results in terms of anomaly detection accuracy. Contribution/Originality: This study proposed multi-classifier model for increasing anomaly detection accuracy in network traffic. The model consists of the J48, LogitBoost, IBk, AdaBoost, RandomTree classifiers. This work performed a comparative analysis of used classifiers and their combination to see which one will give the best result In study classifiers and their combination have been implemented on NSL-KDD open source dataset using WEKA tool. The results show that the ensemble classifiers provide the better result than using these classifiers individually. The computer network traffic analysis with employment of our model can help network engineers and administrators to create a more reliable network, avoid possible discharges and take precautionary measures.

show abstract

“…Anomaly detection is a process of discovering data points or events that significantly deviate from the normal data point or events [3]. This definition specifies that an anomaly detection algorithm must learn normal data points, and it labels these points as anomalous if it deviates significantly from the rest of the data.…”

Section: Anomaly Detectionmentioning

confidence: 99%

Superimposed Rule-Based Classification Algorithm (SRBCA) for One-Class Multivariate Conditional Anomaly Detection

2019

Proceedings of 2019 the 9th International Workshop on Computer Science and Engineering

View full text Add to dashboard Cite

Traditional anomaly detection causes a problem of detecting too numerous false positives in many problem domains. In this work, a Superimpose Rule-Based Classification algorithm (SRBCA) is proposed for conditional anomaly detection. The algorithm is an enhancement of the traditional OneR algorithm. The traditional OneR can generate a set of rules from its attributes with multiple classes, compute the error rate and apply the rule to the attribute with the smallest error. However, OneR has a disadvantage for one-class datasets which contains values belonging to the normal class. The enhanced algorithm, SRBCA, does not embody very complex rules similar to its predecessor. Furthermore, SRBCA includes the generation and application of rules from the one-class dataset in an n-dimensional space using classification. Holdout method was used to evaluate the performance of the classifiers' accuracy which involved training multiple subsets' behavioral and indicator attributes, superimposing rules and testing by using balanced and unbalanced class data to detect and label conditional anomaly data points. This paper shows the comparison between SRBCA, One-Class Support Vector Machine (OCSVM) and other anomaly detection classification algorithms for conditional anomaly detection. It proves that the new method can handle one-class multivariate for conditional anomaly detection with better accuracy.

show abstract

An Anomaly Detection Based on Optimization

Cited by 19 publications

References 45 publications

Efficient algorithm for big data clustering on single machine

Efficient algorithm for big data clustering on single machine

Classification Ensemble Based Anomaly Detection in Network Traffic

Superimposed Rule-Based Classification Algorithm (SRBCA) for One-Class Multivariate Conditional Anomaly Detection

Contact Info

Product

Resources

About