An Efficient Augmented Lagrangian-Based Method for Linear Equality-Constrained Lasso

Abstract

Variable selection is one of the most important tasks in statistics and machine learning. To incorporate more prior information about the regression coefficients, the constrained Lasso model has been proposed in the literature. In this paper, we present an inexact augmented Lagrangian method to solve the Lasso problem with linear equality constraints. By fully exploiting second-order sparsity of the problem, we are able to greatly reduce the computational cost and obtain highly efficient implementations. Furthermore, numerical results on both synthetic data and real data show that our algorithm is superior to existing first-order methods in terms of both running time and solution accuracy.Set J = {j : |x j | > σλ} with cardinality |J | = r. By utilizing the diagonal structure of Q, we can writē

A decomposition method for lasso problems with zero-sum constraint

“…and N k be defined as in (13), with π k computed as in (15) and µ(x k ) computed as in (14). Then, for any optimal solution x * of problem (1), there exists a neighborhood B(x * ) such that…”

“…In this fashion, an augmented Lagrangian scheme with the subproblem solved by cyclic coordinate descent was proposed in [28], while a coordinate descent strategy based on random selection of variables was proposed in [3]. Moreover, considering more general forms of constrained lasso, an approach based on quadratic programming and an ADMM method were analyzed in [18], a semismooth Newton augmented Lagrangian method was proposed in [15] and path algorithms were designed in [18,24,40].…”