The Concentration of Fractional Distances

François, Damien; Wertz, Vincent; Verleysen, Michel

doi:10.1109/tkde.2007.1037

Cited by 260 publications

(217 citation statements)

References 34 publications

Supporting

Mentioning

204

Contrasting

Unclassified

Order By: Relevance

“…However, the analysis of [24] applies when dimensional values are comparable in their extent and exhibit the same properties as for normalized data.…”

Section: Data Setsmentioning

confidence: 99%

“…Although their underlying data models do generally assume (explicitly or otherwise) different underlying mechanisms for the formation of data groupings, they motivate their new approaches with only a passing reference to the curse of dimensionality. Indeed, it has been observed recently that many questions regarding these effects remain open [24]. Thus, a more detailed study of the effects of the curse of dimensionality on such heterogeneously distributed data sets in the presence of both relevant and irrelevant features is needed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Can Shared-Neighbor Distances Defeat the Curse of Dimensionality?

Houle

Kriegel

Kröger

et al. 2010

Lecture Notes in Computer Science

213

153

View full text Add to dashboard Cite

Abstract. The performance of similarity measures for search, indexing, and data mining applications tends to degrade rapidly as the dimensionality of the data increases. The effects of the so-called 'curse of dimensionality' have been studied by researchers for data sets generated according to a single data distribution. In this paper, we study the effects of this phenomenon on different similarity measures for multiplydistributed data. In particular, we assess the performance of sharedneighbor similarity measures, which are secondary similarity measures based on the rankings of data objects induced by some primary distance measure. We find that rank-based similarity measures can result in more stable performance than their associated primary distance measures.

show abstract

“…However, the analysis of [24] applies when dimensional values are comparable in their extent and exhibit the same properties as for normalized data.…”

Section: Data Setsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Can Shared-Neighbor Distances Defeat the Curse of Dimensionality?

Houle

Kriegel

Kröger

et al. 2010

Lecture Notes in Computer Science

213

153

View full text Add to dashboard Cite

show abstract

“…This leads to bad density estimates for highdimensional data, causing difficulties for density-based approaches. The latter is a somewhat counterintuitive property of high-dimensional data representations, where all distances between data points tend to become harder to distinguish as dimensionality increases, which can cause problems with distance-based algorithms [6], [7], [8], [9].…”

Section: Introductionmentioning

confidence: 99%

A Novel Collective Neighbor Clustering in High Dimensional Data

DHA.N¹,

ARULRAJ²

2015

IJIRCCE

View full text Add to dashboard Cite

Clustering becomes difficult due to the increasing sparsity of such data, as well as the increasing difficulty in distinguishing distances between data points. The proposed method called "kernel trick" and "Collective Neighbor Clustering", which takes as input measures of correspondence between pairs of data points. Real-valued hubs are exchanged between data points until a high-quality set of patterns and corresponding clusters gradually emerges. To validate our theory by demonstrating that hubness is a high-quality measure of point centrality within a highdimensional information cluster, and by proposing several hubness-based clustering algorithms, showing that main hubs can be used effectively as cluster prototypes or as guides during the search for centroid-based cluster patterns. Experimental results demonstrate the good performance of our proposed algorithms in manifold settings, mainly focused on large quantities of overlapping noise. The proposed methods are modified mostly for detecting approximately hyper spherical clusters and need to be extended to properly handle clusters of arbitrary shapes.

show abstract

“…The latter arise mostly from a number of phenomena occurring in data sets of this type, known in literature as "the curse of multidimensionality". Above all, this includes the exponential growth in sample size necessary to achieve appropriate effectiveness of data analysis methods with increasing dimension (the empty space phenomenon), as well as the vanishing difference between near and far points (norm concentration) using standard Minkowski distances [2].…”

Section: Introductionmentioning

confidence: 99%

Reduction of Dimension and Size of Data Set by Parallel Fast Simulated Annealing

Kulczycki

Łukasik

2014

Studies in Computational Intelligence

View full text Add to dashboard Cite

Abstract.A universal method of dimension and sample size reduction, designed for exploratory data analysis procedures, constitutes the subject of this paper. The dimension is reduced by applying linear transformation, with the requirement that it has the least possible influence on the respective locations of sample elements. For this purpose an original version of the heuristic Parallel Fast Simulated Annealing method was used. In addition, those elements which change the location significantly as a result of the transformation, may be eliminated or assigned smaller weights for further analysis. As well as reducing the sample size, this also improves the quality of the applied methodology of knowledge extraction. Experimental research confirmed the usefulness of the procedure worked out in a broad range of problems of exploratory data analysis such as clustering, classification, identification of outliers and others.

show abstract

The Concentration of Fractional Distances

Cited by 260 publications

References 34 publications

Can Shared-Neighbor Distances Defeat the Curse of Dimensionality?

Can Shared-Neighbor Distances Defeat the Curse of Dimensionality?

A Novel Collective Neighbor Clustering in High Dimensional Data

Reduction of Dimension and Size of Data Set by Parallel Fast Simulated Annealing

Contact Info

Product

Resources

About