Distributed and scalable Sybil identification based on nearest neighbour approximation using big data analysis techniques

Valliyammai, C.; Devakunchari, R.

doi:10.1007/s10586-018-2314-9

Cited by 7 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…And the validity of the algorithm is analyzed and the final results are calculated by the trained test classification. e essence of cluster analysis is an unsupervised classification process, which is the same as classification algorithms in that they both result in dividing data objects into several categories, with the difference that the data objects of cluster analysis are unlabeled and the category information is unknown [15]. e clustering analysis algorithm is based on the data characteristics of the objects and calculates the similarity between the objects by a specific similarity measure so that similar data objects are grouped into the same cluster and data objects with less similarity are grouped into different clusters.…”

Section: Analysis Of Clustering Algorithms For Bigmentioning

confidence: 99%

Clustering Algorithm for Big Datasets with Mixed Attribute Features under Spark

Wang¹

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

This paper presents an in-depth study and analysis of large datasets of mixed and attribute features under Spark using a large dataset clustering algorithm. The classical algorithm K-means based on division and the density-based clustering algorithm DPC, which has become more popular in recent years, are selected as the research objects of this paper. Secondly, the original K-means algorithm is improved by combining holdout validation and K-means++ method to address the shortcomings of the K-means algorithm that the number of class clusters K needs to be set in advance and the initial class cluster centers are chosen randomly, which leads to unstable iterations and slow convergence of clustering results. The similarity matrix will be continuously updated during the iterative process. It mainly refers to the process of dividing objects into multiple classes according to the degree of similarity between objects. After the division, the objects within the class are like each other, while the objects between the classes are different from each other. The comparison experiments of the improved algorithm before and after the MovieLens dataset are conducted to verify that the new algorithm has better performance in terms of clustering accuracy and efficiency. Again, to address the drawback that the clustering results in the DPC algorithm rely heavily on the subjective selection of the truncation distance parameter cd, and it is difficult to handle datasets with complex distribution and large density variation, the algorithm can generate the optimal cd adaptively by combining K-nearest neighbors and introducing the distance comparison quantity, which has a better performance by considering the overall and local distribution of the data. The feasibility of the improved method is verified by validating the algorithm with artificial datasets and UCI datasets as well as separation tests. Finally, the parallelized design and implementation of the improved K-means algorithm and CDPC-KNN algorithm are completed by building a Spark clustering environment, and the parallelized algorithm is verified to have much better data processing capability and be more adaptable to the clustering analysis of large-scale data by comparing algorithm string parallelism experiments.

show abstract

Section: Analysis Of Clustering Algorithms For Bigmentioning

confidence: 99%

Clustering Algorithm for Big Datasets with Mixed Attribute Features under Spark

Wang¹

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…2017 ; Gilani et al. 2017 ), AdaBoost (Abu-El-Rub and Mueen 2019 ; Andriotis and Takasu 2018 ), K-nearest neighbor (KNN) (Valliyammai and Devakunchari 2019 ) and support vector machine (SVM) (Dorri and Dadfarnia 2018 ).…”

Section: Literature Reviewmentioning

confidence: 99%