Rejoinder: Sparse Regression: Scalable Algorithms and Empirical Performance

Bertsimas, Dimitris; Pauphilet, Jean; Parys, Bart Van

doi:10.1214/20-sts701rej

Cited by 5 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the best-subsets approach could be used for obtaining an initial solution to glasso, as an alternative to the l 1 -regulaized node-wise procedure of Meinshausen and Bühlmann (2006). Alternatively, the methods pioneered by Bertsimas and colleagues (Bertsimas & King, 2016; Bertsimas et al, 2016; Bertsimas et al, 2020a, 2020b) could ultimately become direct competitors for glasso.…”

Section: Discussionmentioning

confidence: 99%

“…The n = 125 setting is a smaller sample size than what is found in most applications in the network psychometrics literature, whereas n = 2,000 is comparable to the sample sizes of the two empirical examples. As noted by Bertsimas et al (2020a), good variable selection procedures should converge toward perfect sensitivity and specificity as sample size increases. Thus, we anticipate significant improvement in sensitivity and specificity as n increases over the range from 125 to 2,000.…”

Section: Design Featuresmentioning

confidence: 99%

“…A recent exchange in the November 2020 issue of Statistical Science highlighted the relative merits and limitations of several variants of the best-subsets and lasso approaches to regression (Bertsimas et al, 2020a(Bertsimas et al, , 2020bChen et al, 2020;Hastie et al, 2020). There was no consensus from the exchange that any single method was always best from a predictive standpoint.…”

Section: Relative Importance Networkmentioning

confidence: 99%

See 2 more Smart Citations

A modified approach to fitting relative importance networks.

Brusco¹,

Watts²,

Steinley³

2024

Psychological Methods

View full text Add to dashboard Cite

Most researchers have estimated the edge weights for relative importance networks using a well-established measure of general dominance for multiple regression. This approach has several desirable properties including edge weights that represent R2 contributions, in-degree centralities that correspond to R2 for each item when using other items as predictors, and strong replicability. We endorse the continued use of relative importance networks and believe they have a valuable role in network psychometrics. However, to improve their utility, we introduce a modified approach that uses best-subsets regression as a preceding step to select an appropriate subset of predictors for each item. The benefits of this modification include: (a) computation time savings that can enable larger relative importance networks to be estimated, (b) a principled approach to edge selection that can significantly improve specificity, (c) the provision of a signed network if desired, (d) the potential use of the best-subsets regression approach for estimating Gaussian graphical models, and (e) possible generalization to best-subsets logistic regression for Ising models. We describe, evaluate, and demonstrate the proposed approach and discuss its strengths and limitations.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Design Featuresmentioning

confidence: 99%

Section: Relative Importance Networkmentioning

confidence: 99%

See 1 more Smart Citation

A modified approach to fitting relative importance networks.

Brusco¹,

Watts²,

Steinley³

2024

Psychological Methods

View full text Add to dashboard Cite

show abstract

“…A future direction of study will be to develop an efficient algorithm specialized for solving our MIQO problem. We are now working on extending several MIO-based high-performance algorithms [24,48,49] to sparse Poisson regression. Another direction of future research is to improve the performance of our methods for selecting tangent lines.…”

Section: Plos Onementioning

confidence: 99%

Sparse Poisson regression via mixed-integer optimization

2021

View full text Add to dashboard Cite

We present a mixed-integer optimization (MIO) approach to sparse Poisson regression. The MIO approach to sparse linear regression was first proposed in the 1970s, but has recently received renewed attention due to advances in optimization algorithms and computer hardware. In contrast to many sparse estimation algorithms, the MIO approach has the advantage of finding the best subset of explanatory variables with respect to various criterion functions. In this paper, we focus on a sparse Poisson regression that maximizes the weighted sum of the log-likelihood function and the L2-regularization term. For this problem, we derive a mixed-integer quadratic optimization (MIQO) formulation by applying a piecewise-linear approximation to the log-likelihood function. Optimization software can solve this MIQO problem to optimality. Moreover, we propose two methods for selecting a limited number of tangent lines effective for piecewise-linear approximations. We assess the efficacy of our method through computational experiments using synthetic and real-world datasets. Our methods provide better log-likelihood values than do conventional greedy algorithms in selecting tangent lines. In addition, our MIQO formulation delivers better out-of-sample prediction performance than do forward stepwise selection and L1-regularized estimation, especially in low-noise situations.

show abstract

“…This 0 norm constrained MIO problem is non-convex and N P-hard [22], corresponding to the best subset selection in the larger statistics community [21,2]. The N P-hardness of the problem has contributed to the belief that discrete optimization problems were intractable [3]. For this reason, plenty of impressive sparsity-promoting techniques have focused on computationally feasible algorithms for solving the approximations, including Lasso [33], Elastic-net [37], nonconvex regularization [15,19] and stepwise regression [13].…”

Section: Introductionmentioning

confidence: 99%