Unsupervised Feature Selection by Pareto Optimization

Feng

2020

AAAI

Self Cite

Subset selection, i.e., to select a limited number of items optimizing some given objective function, is a fundamental problem with various applications such as unsupervised feature selection and sparse regression. By employing a multi-objective evolutionary algorithm (EA) with mutation only to optimize the given objective function and minimize the number of selected items simultaneously, the recently proposed POSS algorithm achieves state-of-the-art performance for subset selection. In this paper, we propose the PORSS algorithm by incorporating recombination, a characterizing feature of EAs, into POSS. We prove that PORSS can achieve the optimal polynomial-time approximation guarantee as POSS when the objective function is monotone, and can find an optimal solution efficiently in some cases whereas POSS cannot. Extensive experiments on unsupervised feature selection and sparse regression show the superiority of PORSS over POSS. Our analysis also theoretically discloses that recombination from diverse solutions can be more likely than mutation alone to generate various variations, thereby leading to better exploration; this may be of independent interest for understanding the influence of recombination.

Section: Empirical Studysupporting

confidence: 82%

Section: The Poss Algorithmmentioning

confidence: 99%

Section: Empirical Studymentioning

confidence: 99%

See 1 more Smart Citation

Subset Selection by Pareto Optimization with Recombination

Feng

2020

AAAI

Self Cite

“…where both the objective function f : 2 V → R and the cost function c : 2 V → R are monotone, but not necessarily submodular. This problem is NP-hard in general, and has various applications, such as influence maximization [Kempe et al, 2003], sensor placement [Krause et al, 2008], document summarization [Lin and Bilmes, 2011] and unsupervised feature selection [Feng et al, 2019], just to name a few. A well-known special case of this problem is subset selection with cardinality constraints, that is, c(X) = |X|.…”

Section: Introductionmentioning

confidence: 99%

Fast Pareto Optimization for Subset Selection with Dynamic Cost Constraints

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

Neumann

et al. 2021

Self Cite

Subset selection with cost constraints is a fundamental problem with various applications such as influence maximization and sensor placement. The goal is to select a subset from a ground set to maximize a monotone objective function such that a monotone cost function is upper bounded by a budget. Previous algorithms with bounded approximation guarantees include the generalized greedy algorithm, POMC and EAMC, all of which can achieve the best known approximation guarantee. In real-world scenarios, the resources often vary, i.e., the budget often changes over time, requiring the algorithms to adapt the solutions quickly. However, when the budget changes dynamically, all these three algorithms either achieve arbitrarily bad approximation guarantees, or require a long running time. In this paper, we propose a new algorithm FPOMC by combining the merits of the generalized greedy algorithm and POMC. That is, FPOMC introduces a greedy selection strategy into POMC. We prove that FPOMC can maintain the best known approximation guarantee efficiently.

“…Besides relaxing the constraints, non-submodular objective functions have also been studied, e.g., (Bian et al 2017;Elenberg et al 2018;Bogunovic, Zhao, and Cevher 2018;Qian et al 2019). They have many applications, such as Bayesian experimental design (Krause, Singh, and Guestrin 2008), dictionary selection (Krause and Cevher 2010), sparse regression (Das and Kempe 2011), and unsupervised feature selection (Feng, Qian, and Tang 2019).…”

Section: Introductionmentioning

confidence: 99%

An Efficient Evolutionary Algorithm for Subset Selection with General Cost Constraints

Feng

et al. 2020

AAAI

Self Cite

In this paper, we study the problem of selecting a subset from a ground set to maximize a monotone objective function f such that a monotone cost function c is bounded by an upper limit. State-of-the-art algorithms include the generalized greedy algorithm and POMC. The former is an efficient fixed time algorithm, but the performance is limited by the greedy nature. The latter is an anytime algorithm that can find better subsets using more time, but without any polynomial-time approximation guarantee. In this paper, we propose a new anytime algorithm EAMC, which employs a simple evolutionary algorithm to optimize a surrogate objective integrating f and c. We prove that EAMC achieves the best known approximation guarantee in polynomial expected running time. Experimental results on the applications of maximum coverage, influence maximization and sensor placement show the excellent performance of EAMC.