Unsupervised Feature Selection via Unified Trace Ratio Formulation and K-means Clustering (TRACK)

Abstract

Feature selection plays a crucial role in scientific research and practical applications. In the real world applications, labeling data is time and labor consuming. Thus, unsupervised feature selection methods are desired for many practical applications. Linear discriminant analysis (LDA) with trace ratio criterion is a supervised dimensionality reduction method that has shown good performance to improve classifications. In this paper, we first propose a unified objective to seamlessly accommodate trace ratio formulation and K-means clustering procedure, such that the trace ratio criterion is extended to unsupervised model. After that, we propose a novel unsupervised feature selection method by integrating unsupervised trace ratio formulation and structured sparsity-inducing norms regularization. The proposed method can harness the discriminant power of trace ratio criterion, thus it tends to select discriminative features. Meanwhile, we also provide two important theorems to guarantee the unsupervised feature selection process. Empirical results on four benchmark data sets show that the proposed method outperforms other sate-of-the-art unsupervised feature selection algorithms in all three clustering evaluation metrics.

Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization

“…Ye et al introduced a kernelized K-means algorithm, denoted by DisKmeans, where embedding to a lower dimensional subspace via linear discriminant analysis (LDA) is jointly learned with K-means cluster assignments [62]. [49] proposed to a new method to simultaneously conduct both clustering and feature embedding/selection tasks to achieve better performance. But these models suffer from having shallow and linear embedding functions, which cannot represent the non-linearity of real-world data.…”

Fast Adaptive K-Means Subspace Clustering for High-Dimensional Data

“…To preserve the original features in the low-dimensional feature space, Wang et al [30] proposed a discriminative learning method with Trace Ratio Formulation and K-means Clustering (TRACK), which jointly unifies trace ratio LDA, K-means, and regularization feature learning into a single objective function. However, similar to LDAKM, TRACK still suffers from ''small-sample-size'' problem and high computational complexity problem.…”

“…Besides, it imposes a balance parameter to control the contributions of within scatter matrix and between scatter matrix [24]. Following the previous works [24], [30], to achieve a fair comparison, all the parameters (if any) of the compared algorithms are tuned by a ''grid-search'' strategy from }, and the best clustering results are recorded with the optimal parameters. We repeat the experiment 50 times independently and report the average results together with the variance.…”

“…In addition, they also neglect the discriminative information among data. Wang et al [30] implemented a one-stage KM type subspace clustering with trace ratio LDA and feature learning. Nevertheless, it also depends on the eigenvalue decomposition.…”

Allocating Bandwidth Based on Users’ Web Usage Patterns for University Networks: A Hybrid Data Mining Approach

“…There were several ways to manage the unlabeled dataset most common method approach is making LDA unsupervised by utilizing clustering algorithms [9,[14][15][16]. With that, SelfOrganizing Map (SOM) was employed in creating the clusters or making of the class labels because it is an unsupervised learning technique to discover patterns to the dataset and it can deal with high dimensional data [17].…”