Feature subset selection for logistic regression via mixed integer optimization

Sato, Toshiki; Takano, Yasunari; Miyashiro, Ryuhei; Yoshise, Akiko

doi:10.1007/s10589-016-9832-2

Cited by 48 publications

(54 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Optimization problems where sparse solutions are sought arise frequently in modern science and engineering. Just as examples, applications of sparse optimization regard compressed sensing in signal processing [1,2], best subset selection [3][4][5][6] and sparse inverse covariance estimation [7,8] in statistics, sparse portfolio selection [9] in decision science, neural networks compression in machine learning [10,11].…”

Section: Introductionmentioning

confidence: 99%

Convergent Inexact Penalty Decomposition Methods for Cardinality-Constrained Problems

Lapucci

Levato

Sciandrone

2020

J Optim Theory Appl

View full text Add to dashboard Cite

In this manuscript, we consider the problem of minimizing a smooth function with cardinality constraint, i.e., the constraint requiring that the "Equation missing"-norm of the vector of variables cannot exceed a given threshold value. A well-known approach of the literature is represented by the class of penalty decomposition methods, where a sequence of penalty subproblems, depending on the original variables and new variables, are inexactly solved by a two-block decomposition method. The inner iterates of the decomposition method require to perform exact minimizations with respect to the two blocks of variables. The computation of the global minimum with respect to the original variables may be prohibitive in the case of nonconvex objective function. In order to overcome this nontrivial issue, we propose a modified penalty decomposition method, where the exact minimizations with respect to the original variables are replaced by suitable line searches along gradient-related directions. We also present a derivative-free penalty decomposition algorithm for black-box optimization. We state convergence results of the proposed methods, and we report the results of preliminary computational experiments.

show abstract

Section: Introductionmentioning

confidence: 99%

Convergent Inexact Penalty Decomposition Methods for Cardinality-Constrained Problems

Lapucci

Levato

Sciandrone

2020

J Optim Theory Appl

View full text Add to dashboard Cite

show abstract

“…Since proper variable selection is essential for data analysis, it has a great advantage over heuristic methods. A direction of future research will be to extend our formulation to classification algorithms (e.g., Sato, Takano, Miyashiro, & Yoshise, 2015).…”

Section: Resultsmentioning

confidence: 99%

Mixed integer second-order cone programming formulations for variable selection in linear regression

Miyashiro

Takano

2015

European Journal of Operational Research

Self Cite

View full text Add to dashboard Cite

“…One of these approaches was first proposed in the 1970s [3], and recently they have received renewed attention due to advances in algorithms and hardware [8], [16]. The MIO approaches have recently been used effectively for linear regression [14], [19], [20], logistic regression [7], [23], support vector machine [18], classification tree [5], and various applications [29], [30].…”

Section: Introductionmentioning

confidence: 99%

Feature Subset Selection for Ordered Logit Model via Tangent-Plane-Based Approximation

Naganuma

Takano

Miyashiro

2019

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

This paper is concerned with a mixed-integer optimization (MIO) approach to selecting a subset of relevant features from among many candidates. For ordinal classification, a sequential logit model and an ordered logit model are often employed. For feature subset selection in the sequential logit model, Sato et al. [22] recently proposed a mixed-integer linear optimization (MILO) formulation. In their MILO formulation, a univariate nonlinear function contained in the sequential logit model was represented by a tangent-line-based approximation. We extend this MILO formulation toward the ordered logit model, which is more commonly used for ordinal classification than the sequential logit model is. Making use of tangent planes to approximate a bivariate nonlinear function involved in the ordered logit model, we derive an MILO formulation for feature subset selection in the ordered logit model. Our computational results verify that the proposed method is superior to the L1-regularized ordered logit model in terms of solution quality.

show abstract

Feature subset selection for logistic regression via mixed integer optimization

Cited by 48 publications

References 30 publications

Convergent Inexact Penalty Decomposition Methods for Cardinality-Constrained Problems

Convergent Inexact Penalty Decomposition Methods for Cardinality-Constrained Problems

Mixed integer second-order cone programming formulations for variable selection in linear regression

Feature Subset Selection for Ordered Logit Model via Tangent-Plane-Based Approximation

Contact Info

Product

Resources

About