Spectral Clustering Approach with K-Nearest Neighbor and Weighted Mahalanobis Distance for Data Mining

Yin, Lifeng; Lv, Lei; Wang, Dingyi; Qu, Yingwei; Chen, Huayue; Deng, Wu

doi:10.3390/electronics12153284

Cited by 6 publications

(2 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Mahalanobis distance and k-means were also used in [22] to improve the clustering accuracy for Laplacian matrix eigenvectors.…”

Section: Hierarchical Models For Vision Tasksmentioning

confidence: 99%

Hierarchical Classification for Large-Scale Learning

Wang,

Barbu

2023

Electronics

View full text Add to dashboard Cite

Deep neural networks (DNNs) have drawn much attention due to their success in various vision tasks. Current DNNs are used on data with a relatively small number of classes (e.g., 1000 or less) and employ a fully connected layer for classification, which allocates one neuron for each class and thus, per-example, the classification scales as O(K) with the number of classes K. This approach is computationally intensive for many real-life applications where the number of classes is very large (e.g., tens of thousands of classes). To address this problem, our paper introduces a hierarchical approach for classification with a large number of classes that scales as O(K) and could be extended to O(logK) with a deeper hierarchy. The method, called Hierarchical PPCA, uses a self-supervised pretrained feature extractor to obtain meaningful features and trains Probabilistic PCA models on the extracted features for each class separately, making it easy to add classes without retraining the whole model. The Mahalanobis distance is used to obtain the classification result. To speed-up classification, the proposed Hierarchical PPCA framework clusters the image class models, represented as Gaussians, into a smaller number of super-classes using a modified k-means clustering algorithm. The classification speed increase is obtained by Hierarchical PPCA assigning a sample to a small number of the most likely super-classes and restricting the image classification to the image classes corresponding to these super-classes. The fact that the model is trained on each class separately makes it applicable to training on very large datasets such as the whole ImageNet with more than 10,000 classes. Experiments on three standard datasets (ImageNet-100, ImageNet-1k,and ImageNet-10k) indicate that the hierarchical classifier can achieve a superior accuracy with up to a 16-fold speed increase compared to a standard fully connected classifier.

show abstract

“…The Mahalanobis distance and k-means were also used in [22] to improve the clustering accuracy for Laplacian matrix eigenvectors.…”

Section: Hierarchical Models For Vision Tasksmentioning

confidence: 99%

Hierarchical Classification for Large-Scale Learning

Wang,

Barbu

2023

Electronics

View full text Add to dashboard Cite

show abstract

“…Since the effective information in high-dimensional data usually resides in low-dimensional structures, many subspace clustering methods have been proposed. These subspace-based clustering methods have proven to be effective in mining feature information from high-dimensional data and are widely applied in handling computer vision tasks [4,5].…”

Section: Introductionmentioning

confidence: 99%

Large-Scale Subspace Clustering Based on Purity Kernel Tensor Learning

Zheng,

Zhao,

Zhang

et al. 2023

Electronics

View full text Add to dashboard Cite

In conventional subspace clustering methods, affinity matrix learning and spectral clustering algorithms are widely used for clustering tasks. However, these steps face issues, including high time consumption and spatial complexity, making large-scale subspace clustering (LS2C) tasks challenging to execute effectively. To address these issues, we propose a large-scale subspace clustering method based on pure kernel tensor learning (PKTLS2C). Specifically, we design a pure kernel tensor learning (PKT) method to acquire as much data feature information as possible while ensuring model robustness. Next, we extract a small sample dataset from the original data and use PKT to learn its affinity matrix while simultaneously training a deep encoder. Finally, we apply the trained deep encoder to the original large-scale dataset to quickly obtain its projection sparse coding representation and perform clustering. Through extensive experiments on large-scale real datasets, we demonstrate that the PKTLS2C method outperforms existing LS2C methods in clustering performance.

show abstract