FINDIT: a fast and intelligent subspace clustering algorithm using dimension voting

Woo, Kyoung Gu; Lee, Jeong Hoon; Kim, Myoung Ho; Lee, Yoon Joon

doi:10.1016/j.infsof.2003.07.003

Cited by 103 publications

(54 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the demonstrated deficiency of conventional L p norms for high-dimensional data, a plethora of work based on the Euclidean distance has been dedicated to clustering strategies, which appear to be effective in practice to varying degrees for high-dimensional data [5]. Many heuristics have recently been proposed or evaluated for clustering [6][7][8][9][10][11][12][13][14], outlier detection [15][16][17][18], and indexing or similarity search [6,[19][20][21][22][23]] that seek to mitigate the effects of the curse of dimensionality. While some of these strategies, such as projected or subspace clustering, do recognize implicitly the effect of relevant versus irrelevant attributes for a cluster, all these papers (as well as others) abstain from discussing these effects, let alone studying them in detail.…”

Section: Introductionmentioning

confidence: 99%

Can Shared-Neighbor Distances Defeat the Curse of Dimensionality?

Houle

Kriegel

Kröger

et al. 2010

Lecture Notes in Computer Science

213

153

View full text Add to dashboard Cite

Abstract. The performance of similarity measures for search, indexing, and data mining applications tends to degrade rapidly as the dimensionality of the data increases. The effects of the so-called 'curse of dimensionality' have been studied by researchers for data sets generated according to a single data distribution. In this paper, we study the effects of this phenomenon on different similarity measures for multiplydistributed data. In particular, we assess the performance of sharedneighbor similarity measures, which are secondary similarity measures based on the rankings of data objects induced by some primary distance measure. We find that rank-based similarity measures can result in more stable performance than their associated primary distance measures.

show abstract

Section: Introductionmentioning

confidence: 99%

Can Shared-Neighbor Distances Defeat the Curse of Dimensionality?

Houle

Kriegel

Kröger

et al. 2010

Lecture Notes in Computer Science

213

153

View full text Add to dashboard Cite

show abstract

“…In top down approaches, the user defines the number of clusters and the number of relevant subspaces [30,31]. It is not possible to automatically find all possible clusters in all the subspaces using this method.…”

Section: Motivating Examplesmentioning

confidence: 99%

“…Top down approaches like PROCLUS [30] and FINDIT [31] use projected clustering for high-dimensional data. An initial approximation of the number of the clusters and the relevant subspaces are given by the user.…”

Section: Background and Literature Reviewmentioning

confidence: 99%

A novel algorithm for fast and scalable subspace clustering of high-dimensional data

Kaur

Datta

2015

Journal of Big Data

View full text Add to dashboard Cite

Rapid growth of high dimensional datasets in recent years has created an emergent need to extract the knowledge underlying them. Clustering is the process of automatically finding groups of similar data points in the space of the dimensions or attributes of a dataset. Finding clusters in the high dimensional datasets is an important and challenging data mining problem. Data group together differently under different subsets of dimensions, called subspaces. Quite often a dataset can be better understood by clustering it in its subspaces, a process called subspace clustering. But the exponential growth in the number of these subspaces with the dimensionality of data makes the whole process of subspace clustering computationally very expensive. There is a growing demand for efficient and scalable subspace clustering solutions in many Big data application domains like biology, computer vision, astronomy and social networking. Apriori based hierarchical clustering is a promising approach to find all possible higher dimensional subspace clusters from the lower dimensional clusters using a bottom-up process. However, the performance of the existing algorithms based on this approach deteriorates drastically with the increase in the number of dimensions. Most of these algorithms require multiple database scans and generate a large number of redundant subspace clusters, either implicitly or explicitly, during the clustering process. In this paper, we present SUBSCALE, a novel clustering algorithm to find non-trivial subspace clusters with minimal cost and it requires only k database scans for a k-dimensional data set. Our algorithm scales very well with the dimensionality of the dataset and is highly parallelizable. We present the details of the SUBSCALE algorithm and its evaluation in this paper.

show abstract

“…However, the subspace in which each cluster is embedded is not explicitly known from the algorithm. The FINDIT (fast and intelligent subspace clustering algorithm using dimension voting) [31] algorithm, uses a dimension voting technique to find subspace clusters. Dimension oriented distance is defined to measure the distance between points based on not only the value information but also the dimension information.…”

Section: Related Workmentioning

confidence: 99%

Subspace Clustering of High-Dimensional Data: An Evolutionary Approach

Singh

Sahoo

2013

Applied Computational Intelligence and Soft Computing

View full text Add to dashboard Cite

Clustering high-dimensional data has been a major challenge due to the inherent sparsity of the points. Most existing clustering algorithms become substantially inefficient if the required similarity measure is computed between data points in the fulldimensional space. In this paper, we have presented a robust multi objective subspace clustering (MOSCL) algorithm for the challenging problem of high-dimensional clustering. The first phase of MOSCL performs subspace relevance analysis by detecting dense and sparse regions with their locations in data set. After detection of dense regions it eliminates outliers. MOSCL discovers subspaces in dense regions of data set and produces subspace clusters. In thorough experiments on synthetic and real-world data sets, we demonstrate that MOSCL for subspace clustering is superior to PROCLUS clustering algorithm. Additionally we investigate the effects of first phase for detecting dense regions on the results of subspace clustering. Our results indicate that removing outliers improves the accuracy of subspace clustering. The clustering results are validated by clustering error (CE) distance on various data sets. MOSCL can discover the clusters in all subspaces with high quality, and the efficiency of MOSCL outperforms PROCLUS.

show abstract

FINDIT: a fast and intelligent subspace clustering algorithm using dimension voting

Cited by 103 publications

References 11 publications

Can Shared-Neighbor Distances Defeat the Curse of Dimensionality?

Can Shared-Neighbor Distances Defeat the Curse of Dimensionality?

A novel algorithm for fast and scalable subspace clustering of high-dimensional data

Subspace Clustering of High-Dimensional Data: An Evolutionary Approach

Contact Info

Product

Resources

About