Weighted Clustering for Anomaly Detection in Big Data

Alguliyev, Rasim; Alıguliyev, Ramiz M.; İmamverdiyev, Yadigar; Sukhostat, Lyudmila

doi:10.19139/soic.v6i2.404

Cited by 5 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A large number of researches have been devoted to big data analysis [3, 4, 6, 7]. The issue with big data clustering is that a lot of memory is required.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Efficient algorithm for big data clustering on single machine

Alguliyev

Alıguliyev

Sukhostat

2020

CAAI trans. intell. technol.

View full text Add to dashboard Cite

Big data analysis requires the presence of large computing powers, which is not always feasible. And so, it became necessary to develop new clustering algorithms capable of such data processing. This study proposes a new parallel clustering algorithm based on the k‐means algorithm. It significantly reduces the exponential growth of computations. The proposed algorithm splits a dataset into batches while preserving the characteristics of the initial dataset and increasing the clustering speed. The idea is to define cluster centroids, which are also clustered, for each batch. According to the obtained centroids, the data points belong to the cluster with the nearest centroid. Real large datasets are used to conduct the experiments to evaluate the effectiveness of the proposed approach. The proposed approach is compared with k‐means and its modification. The experiments show that the proposed algorithm is a promising tool for clustering large datasets in comparison with the k‐means algorithm.

show abstract

“…A large number of researches have been devoted to big data analysis [3, 4, 6, 7]. The issue with big data clustering is that a lot of memory is required.…”

Section: Related Workmentioning

confidence: 99%

“…Clustering algorithms are widely applied to big data analysis to show the internal relationship between the data [1][2][3][4][5][6][7][8]. The most popular among them is k-means.…”

Section: Introductionmentioning

confidence: 99%

Efficient algorithm for big data clustering on single machine

Alguliyev

Alıguliyev

Sukhostat

2020

CAAI trans. intell. technol.

View full text Add to dashboard Cite

show abstract

“…The FCM technique partitions data elements or objects into clusters based on their similarity of behavior [15,16,17]. We have developed our algorithm for verification of signatures with FCM technique on the assumption that if the cluster size (i.e.…”

Section: Fcm Clustering Techniquementioning

confidence: 99%

An Ensemble Based Offline Handwritten Signature Verification System

Deka

Mahanta

2020

Stat., optim. inf. comput.

View full text Add to dashboard Cite

In the field of security and forgery prevention, handwritten signatures are the most widely recognized biometric since long and also most practical. Although handwritten signature verification systems are studied using both On-line and Off-line approaches, Off-line signature verification systems are more difficult to compare to On-line verification systems. This is due to the lack of dynamic information, viz. a database which constantly stores the latest signature of the person. In the paper an approach using ensemble methods are adopted to classify a signature as forgery or not. In proposed system, three classifiers, viz, one unsupervised, viz. Fuzzy C-Means (FCM) and two supervised classifiers, viz. Naive Bayes (NB) and Support Vector Machine (SVM) are used as base classifiers. An attempt is made to compare the different approaches. We attempt both the categories of classification not only because both of them are applicable in this particular case but also with an objective of finding out which performs better. In most cases it is observed that Naive Bayes and Ensemble are comparable as they exhibit better performance than the other two. But among them, in most of the cases Ensemble classifier performs better than the Naive Bayes and consequently we have taken the Ensemble as a final classifier.

show abstract

“…The purpose of the clustering algorithm is to maximize the inter-cluster distance and minimize the intra-cluster distance. The well-known k-means algorithm has been applied to many practical clustering problems [2,3,4,5,6,7,8]. The purpose of this method is to automatically divide the dataset into k groups.…”

Section: Introductionmentioning

confidence: 99%

PSO+K-means Algorithm for Anomaly Detection in Big Data

Alguliyev

Alıguliyev

Abdullayeva

2019

Stat., optim. inf. comput.

View full text Add to dashboard Cite

The use of clustering methods in anomaly detection is considered as an effective approach. The choice of the cluster primary center and the finding of local optimum in the well-known k-means and other classic clustering algorithms are considered as one of the major problems and do not allow to get accurate results in anomaly detection. In this paper to improve the accuracy of anomaly detection based on the combination of PSO (particle swarm optimization) and k-means algorithms, the new weighted clustering method is proposed. The proposed method is tested on Yahoo! S5 dataset and a comparative analysis of the obtained results with the k-means algorithm is performed. The results of experiments show that compared to the k-means algorithm the proposed method is more robust and allows to get more accurate results.

show abstract

Weighted Clustering for Anomaly Detection in Big Data

Cited by 5 publications

References 30 publications

Efficient algorithm for big data clustering on single machine

Efficient algorithm for big data clustering on single machine

An Ensemble Based Offline Handwritten Signature Verification System

PSO+K-means Algorithm for Anomaly Detection in Big Data

Contact Info

Product

Resources

About