Hubness-Based Clustering of High-Dimensional Data

Abstract

Hubness has recently been established as a significant property of k-nearest neighbor (k-NN) graphs obtained from high-dimensional data using a distance measure, with traits and effects relevant to the cluster structure of data, as well as clustering algorithms. The hubness property is manifested with increasing (intrinsic) data dimensionality. The distribution of data point in-degrees, i.e. the number of times points appear among the k nearest neighbors of other points in the data, becomes highly skewed. This results in hub points that can have in-degrees multiple orders of magnitude higher than expected. In this chapter we review and refine existing work which explains the mechanisms of the phenomenon, establishes the location of hub points near central regions of clusters in the data, and shows how hubness can negatively affect existing clustering algorithms by virtue of hub points lowering between-cluster distance. Next, we review the newly proposed partitional clustering algorithms, based on K-means, which take advantage of hubness by employing hubs in the process of cluster prototype selection. These "soft" K-means extensions avoid premature convergence to suboptimal stable cluster configurations and are able to reach the global optima more often. The algorithms offer significant improvements over the K-means baseline in scenarios involving high-dimensional and noisy data. The improvements stem from a better placement of hub points into clusters, which helps in increasing the between-cluster distance. Finally, we introduce novel clustering algorithms as "kernelized" versions of the most successful hubness-based methods discussed above, that are able to more effectively handle arbitrarily-shaped clusters.

Clustering Evaluation in High-Dimensional Data

“…Additionally, since it was demonstrated on several occasions that a better handling of hub points may result in better overall clustering quality in manydimensional problems [40,83,84], we intend to consider either extending the existing clustering quality indexes or proposing new ones that would incorporate this finding into account.…”

“…Hubness-based clustering has recently been proposed for high-dimensional clustering problems [83,84] and has been successfully applied in some domains like document clustering [40].…”

ParkinsoNET: Estimation of UPDRS Score Using Hubness-Aware Feedforward Neural Networks

“…Unfortunately, some of the hubs are bad in the sort of sense that they may mislead machine learning algorithms. The presence of hubs have been studied primarily in context of classification, clustering and instance selection, see (Radovanović et al, 2010a), (Tomašev and Mladenić, 2013), (Radovanović et al, 2009), (Radovanović et al, 2010b), (Tomašev et al, 2011), (Tomašev et al, 2015b), , and (Tomašev et al, 2015a) for a survey.…”

EDSC: Efficient document subspace clustering technique for high-dimensional data

“…A feature selection technique explores the possibility of dimensional subset for carrying out clustering by eliminating unnecessary and inappropriate dimensions. Subspace clustering is one of such technique that positions its search operation and generates information about the clusters present in multiple subspaces in overlapping conditions [4] [5].…”