The Lawson-Hanson Algorithm with Deviation Maximization: Finite Convergence and Sparse Recovery

Dessole, Monica; Dell'Orto, Marco; Marcuzzi, Fabio

doi:10.48550/arxiv.2108.05345

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to the simplex method, the algorithm is an active-set algorithm that iteratively sets parts of the variables to zero in an attempt to identify the active constraints and solves the unconstrained least squares sub-problem for this active set of constraints. It is still, arguably, the most famous method for solving (NNLS) and several improvements have been proposed in a series of follow-up papers [8,66,47,49,21]. Its caveat, however, is that it depends on the normal equations, which makes it infeasible for ill-conditioned or large scale problems.…”

Section: Related Work -Nnlsmentioning

confidence: 99%

“…Its caveat, however, is that it depends on the normal equations, which makes it infeasible for ill-conditioned or large scale problems. Moreover, up to this point there exist no better theoretical guarantees for the algorithm and its modifications than convergence in finitely many steps [44,Chapter 23], [21,Theorem 3]. Another line of research has been developing projected gradient methods for solving (NNLS), which come with linear convergence guarantees [41], [55] [46].…”

Section: Related Work -Nnlsmentioning

confidence: 99%

See 1 more Smart Citation

Non-negative Least Squares via Overparametrization

Chou¹,

Maly²,

Verdun³

2022

Preprint

View full text Add to dashboard Cite

In many applications, solutions of numerical problems are required to be non-negative, e.g., when retrieving pixel intensity values or physical densities of a substance. In this context, nonnegative least squares (NNLS) is a ubiquitous tool, especially when seeking sparse solutions of high-dimensional statistical problems. Despite vast efforts since the seminal work of Lawson and Hanson in the '70s, the non-negativity assumption is still an obstacle for the theoretical analysis and scalability of many off-the-shelf solvers. In the different context of deep neural networks, we recently started to see that the training of overparametrized models via gradient descent leads to surprising generalization properties and the retrieval of regularized solutions. In this paper, we prove that, by using an overparametrized formulation, NNLS solutions can reliably be approximated via vanilla gradient flow. We furthermore establish stability of the method against negative perturbations of the ground-truth. Our simulations confirm that this allows to use vanilla gradient descent as a novel and scalable numerical solver for NNLS.

show abstract