A dimension reduction algorithm preserving both global and local clustering structure

Cai, Weiling

doi:10.1016/j.knosys.2016.11.020

Cited by 24 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This issue is inadequately addressed by older methods due to limitations of their dimensionality reduction algorithms. More recent techniques still experience important limitations in this regard, such as the need for a 'balancing' parameter that may crucially impact the structures preserved (e.g., UMAP, GLSPP [72]), or the imposition of specific metrics that limit their application to other fields (e.g., PHATE, DGL [73]).…”

Section: Discussionmentioning

confidence: 99%

Cluster-based multidimensional scaling embedding tool for data visualization

Hernández-León,

Caro

2024

Phys. Scr.

View full text Add to dashboard Cite

We present a new technique for visualizing high-dimensional data called cluster MDS (cl-MDS), which addresses a common difficulty of dimensionality reduction methods: preserving both local and global structures of the original sample in a single 2-dimensional visualization. Its algorithm combines the well-known multidimensional scaling (MDS) tool with the k-medoids data clustering technique, and enables hierarchical embedding, sparsification and estimation of 2-dimensional coordinates for additional points. While cl-MDS is a generally applicable tool, we also include specific recipes for atomic structure applications. We apply this method to non-linear data of increasing complexity where different layers of locality are relevant, showing a clear improvement in their retrieval and visualization quality.

show abstract

Section: Discussionmentioning

confidence: 99%

Cluster-based multidimensional scaling embedding tool for data visualization

Hernández-León,

Caro

2024

Phys. Scr.

View full text Add to dashboard Cite

show abstract

“…However, this method of reducing data dimensionality is simple to calculate and guarantees the generation of accurate representations of high-dimensional datasets in lower dimensions. The following is the formula for Principal Component Analysis (PCA) [69].…”

Section: Principal Component Analysismentioning

confidence: 99%

K-Hyperparameter Tuning in High-Dimensional Space Clustering: Solving Smooth Elbow Challenges Using an Ensemble Based Technique of a Self-Adapting Autoencoder and Internal Validation Indexes

Gikera,

Mwaura,

Muuro

et al. 2023

Journal on Artificial Intelligence

View full text Add to dashboard Cite

k-means is a popular clustering algorithm because of its simplicity and scalability to handle large datasets. However, one of its setbacks is the challenge of identifying the correct k-hyperparameter value. Tuning this value correctly is critical for building effective k-means models. The use of the traditional elbow method to help identify this value has a long-standing literature. However, when using this method with certain datasets, smooth curves may appear, making it challenging to identify the k-value due to its unclear nature. On the other hand, various internal validation indexes, which are proposed as a solution to this issue, may be inconsistent. Although various techniques for solving smooth elbow challenges exist, k-hyperparameter tuning in high-dimensional spaces still remains intractable and an open research issue. In this paper, we have first reviewed the existing techniques for solving smooth elbow challenges. The identified research gaps are then utilized in the development of the new technique. The new technique, referred to as the ensemble-based technique of a self-adapting autoencoder and internal validation indexes, is then validated in high-dimensional space clustering. The optimal k-value, tuned by this technique using a voting scheme, is a trade-off between the number of clusters visualized in the autoencoder's latent space, k-value from the ensemble internal validation index score and one that generates a value of 0 or close to 0 on the derivativeat the elbow. Experimental results based on the Cochran's Q test, ANOVA, and McNemar's score indicate a relatively good performance of the newly developed technique in k-hyperparameter tuning. KEYWORDS k-hyperparameter tuning; high-dimensional; smooth elbow K-Means ArchitectureK-means is an iterative algorithm that aims to partition a dataset into a set of k non-overlapping groups of data points [9]. The k-hyperparameter is one of the most important hyperparameters to tune in k-means [10,11]. Tuning a machine learning model's hyperparameters has a significant effect on its performance [12]. In this subsection, we explore k-clusters, k-hyperparameters, and the traditional elbow method used to identify the optimal value for the k-hyperparameter. K-Means ClustersThe k-means clusters are the resulting data sub-groups generated by the popular unsupervised partitioning algorithm, known as the k-means clustering algorithm [13]. Cluster analysis using the k-means algorithm is an example of a k-means-based model that has been successfully applied in cluster analysis [14,15]. The k-clusters generated by these algorithms are usually composed of distinct non-overlapping groups of data points, aggregated together because they share specific similarities [16]. The data points within a particular cluster are similar, while the data points across different clusters are different [17]. Both the intra-cluster and inter-cluster data points are measured using a sum of squared distance-based metric [18,19]. For this reason, the original un-partitioned dataset is stan...

show abstract

“…Dimensionality reduction, which projects original features into a lower dimensional space, has been a prevalent technique in dealing with high dimensional datasets, because it is able to remove redundant features, reduce memory usage, avoid the curse of dimensionality and improve efficiency of machine learning algorithm. As a preprocessing step, dimensionality reduction has been applied to a variety of problems including k-means clustering [1,2,3], support vector machines classification [4,5,6,7], k-nearest neighbors classification [8], least squares regression, and low rank approximation [9]. However, how to design efficient and effective dimensionality reduction algorithm is a serious challenge problem.…”

Section: Introductionmentioning

confidence: 99%

Stable sparse subspace embedding for dimensionality reduction

Chen

Zhou

2020

Knowledge-Based Systems

View full text Add to dashboard Cite

Sparse random projection (RP) is a popular tool for dimensionality reduction that shows promising performance with low computational complexity. However, in the existing sparse RP matrices, the positions of non-zero entries are usually randomly selected. Although they adopt uniform sampling with replacement, due to large sampling variance, the number of non-zeros is uneven among rows of the projection matrix which is generated in one trial, and more data information may be lost after dimension reduction. To break this bottleneck, based on random sampling without replacement in statistics, this paper builds a stable sparse subspace embedded matrix (S-SSE), in which non-zeros are uniformly distributed. It is proved that the S-SSE is stabler than the existing matrix, and it can maintain Euclidean distance between points well after dimension reduction.Our empirical studies corroborate our theoretical findings and demonstrate that our approach can indeed achieve satisfactory performance.

show abstract

A dimension reduction algorithm preserving both global and local clustering structure

Cited by 24 publications

References 35 publications

Cluster-based multidimensional scaling embedding tool for data visualization

Cluster-based multidimensional scaling embedding tool for data visualization

K-Hyperparameter Tuning in High-Dimensional Space Clustering: Solving Smooth Elbow Challenges Using an Ensemble Based Technique of a Self-Adapting Autoencoder and Internal Validation Indexes

Stable sparse subspace embedding for dimensionality reduction

Contact Info

Product

Resources

About