Unsupervised Feature Selection via Orthogonal Basis Clustering and Local Structure Preserving

Abstract

Due to the "curse of dimensionality" issue, how to discard redundant features and select informative features in high-dimensional data has become a critical problem, and there are many researches dedicated to solving this problem. Unsupervised feature selection technique, which doesn't require any prior category information to conduct with, has gained a prominent place in pre-processing high-dimensional data among all feature selection techniques, and it has been applied to many neural networks and learning systems related applications, e.g., pattern classification. In this paper, we propose an efficient method for unsupervised feature selection via Orthogonal basis Clustering and reliable Local Structure Preserving, which is referred to OCLSP briefly. Our OCLSP method consists of an orthogonal basis clustering together with an adaptive graph regularization, which realize the functionality of simultaneously achieving excellent cluster separation and preserving the local information of data. Besides, we exploit an efficient alternative optimization algorithm to solve the challenging optimization problem of our proposed OCLSP method, and we perform a theoretical analysis of its computational complexity and convergence. Eventually, we conduct comprehensive experiments on nine real-world datasets to test the validity of our proposed OCLSP method, and the experimental results demonstrate that our proposed OCLSP method outperforms many state-of-the-art unsupervised feature selection methods in terms of clustering accuracy and normalized mutual information, which indicates that our proposed OCLSP method has a strong ability in identifying more important features.

Fast Sparse PCA via Positive Semidefinite Projection for Unsupervised Feature Selection

“…The calculation of a similarity matrix is realized by first constructing a graph. And spectral-based methods can be roughly divided into two types [23]: one to use a predefined graph, while the other learn an adaptive graph during the optimization.…”

“…This inevitably leads to sensitivity to noise features and outlier data. Secondly, many EUFS methods construct a non-convex optimization problem with a series of arXiv:2309.06202v1 [cs.CV] 12 Sep 2023 subproblems to solve in each iteration [22], [23], [31], [32]. Considering the computational complexity of them is usually cubic to the number of samples (or the number of features), the running time of solving the whole problem can sometimes be unacceptable.…”

Joint Multi-view Unsupervised Feature Selection and Graph Learning

“…To improve the orthogonal basis clustering based feature selection method in [18], Lim and Kim [19] incorporated a pairwise dependence term into the objective function. And Lin et al [20] utilized a locality preserving term and a graph regularization term in the orthogonal decomposition based formulation. These methods [15]- [20] are typically designed for single-view unsupervised feature section, but lack the ability to leverage the rich and complementary information in data with multiple views.…”

“…And Lin et al [20] utilized a locality preserving term and a graph regularization term in the orthogonal decomposition based formulation. These methods [15]- [20] are typically designed for single-view unsupervised feature section, but lack the ability to leverage the rich and complementary information in data with multiple views.…”

“…The target matrix T (v) is typically unknown in unsupervised scenarios. To enable the cluster structure learning, the orthogonal decomposition can be employed on the target matrix [18]- [20]. In a simple situation, we can treat each view separately and decompose the target matrix as…”