DBDC: Density Based Distributed Clustering

Januzaj, Eshref; Kriegel, Hans‐Peter; Pfeifle, Martin

doi:10.1007/978-3-540-24741-8_7

Cited by 100 publications

(84 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However it is not fully paralleled while it still needs a single node to aggregate intermediate results. Januzaj et al [7] depict a distributed clustering based on DBSCAN and DBDC and formed local and global two-level clustering. The local clustering is carried out independently on local data; then global clustering is done on a central server based on the transmitted representatives from local clustering.…”

Section: Related Workmentioning

confidence: 99%

“…; (6) if . = Unclassified then (7) if . = Unclassified then (8) Create c-cluster ← ( ); (9) if expandcluster ( , , , , , ) then (10) Create c-cluster ← ( ++); (11) end if (12) end if (13) the cells according to the received points and given in each of the mappers, so the cell with the same in different mappers stands for the same area; thus we can only use the cell to locate the assigned range in overall data space.…”

Section: Cludoop Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

Cludoop: An Efficient Distributed Density-Based Clustering for Big Data Using Hadoop

Zhao

Wang

et al. 2015

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

Density-based clustering for big data is critical for many modern applications ranging from Internet data processing to massivescale moving object management. This paper proposes Cludoop algorithm, an efficient distributed density-based clustering for big data using Hadoop. First, we propose a serial clustering algorithm CluC by leveraging cell partition optimization and c-cluster to fast find clusters. CluC completes classification of the points using the relationships of connected cells around points instead of expensive completed neighbor query, which significantly reduce the number of distance calculations. Second, we propose the Cludoop, which can efficiently cluster very-large-scale data in parallel using already existing data partition on Map/Reduce platform. It employs the proposed serial clustering CluC as a plugged-in clustering on parallel mapper, along with a cell description instead of completed cell in transmission to reduce both network and I/O costs. Guided by proposed cell-based principles, we also design a Merging-Refinement-Merging 3-step framework to merge c-clusters on the overlay of assigned preclustering result on reducer. Finally, our comprehensive experimental evaluation on 10 network-connected commercial PCs, using both huge-volume real and synthetic data, demonstrates (1) the effectiveness of our algorithm in finding correct clusters with arbitrary shape and (2) the fact that our proposed algorithm exhibits better scalability and efficiency than state-of-the-art method.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Cludoop Frameworkmentioning

confidence: 99%

Cludoop: An Efficient Distributed Density-Based Clustering for Big Data Using Hadoop

Zhao

Wang

et al. 2015

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

show abstract

“…An overview of some state-of-the-art research results is given in [10]. Januzaj et al [8] propose a density-based distributed clustering approach. Their recursive technique consists of four different steps, whereas they assume that the data is horizontally distributed.…”

Section: Related Workmentioning

confidence: 99%

Two-phase clustering strategy for gene expression data sets

Habich

Wächter

Lehner

et al. 2006

Proceedings of the 2006 ACM Symposium on Applied Computing

View full text Add to dashboard Cite

In the context of genome research, the method of gene expression analysis has been used for several years. Related microarray experiments are conducted all over the world, and consequently, a vast amount of microarray data sets are produced. Having access to this variety of repositories, researchers would like to incorporate this data in their analyses to increase the statistical significance of their results. In this paper, we present a new two-phase clustering strategy which is based on the combination of local clustering results to obtain a global clustering. The advantage of such a technique is that each microarray data set can be normalized and clustered separately. The set of different relevant local clustering results is then used to calculate the global clustering result. Furthermore, we present an approach based on technical as well as biological quality measures to determine weighting factors for quantifying the local results proportion within the global result. The better the attested quality of the local results, the stronger their impact on the global result.

show abstract

“…In [6,7], density-based distributed clustering algorithms were presented which are based on the density-based partitioning clustering algorithm DBSCAN. The idea of these approaches is to determine suitable local objects representing several other local objects.…”

Section: Related Workmentioning

confidence: 99%

Approximated Clustering of Distributed High-Dimensional Data

Kriegel¹,

Kunath²,

Pfeifle³

et al. 2005

Advances in Knowledge Discovery and Data Mining

Self Cite

View full text Add to dashboard Cite

Abstract. In many modern application ranges high-dimensional feature vectors are used to model complex real-world objects. Often these objects reside on different local sites. In this paper, we present a general approach for extracting knowledge out of distributed data sets without transmitting all data from the local clients to a server site. In order to keep the transmission cost low, we first determine suitable local feature vector approximations which are sent to the server. Thereby, we approximate each feature vector as precisely as possible with a specified number of bytes. In order to extract knowledge out of these approximations, we introduce a suitable distance function between the feature vector approximations. In a detailed experimental evaluation, we demonstrate the benefits of our new feature vector approximation technique for the important area of distributed clustering. Thereby, we show that the combination of standard clustering algorithms and our feature vector approximation technique outperform specialized approaches for distributed clustering when using high-dimensional feature vectors.

show abstract

DBDC: Density Based Distributed Clustering

Cited by 100 publications

References 11 publications

Cludoop: An Efficient Distributed Density-Based Clustering for Big Data Using Hadoop

Cludoop: An Efficient Distributed Density-Based Clustering for Big Data Using Hadoop

Two-phase clustering strategy for gene expression data sets

Approximated Clustering of Distributed High-Dimensional Data

Contact Info

Product

Resources

About