Screening Tests for Lasso Problems

Abstract

Abstract-This paper is a survey of dictionary screening for the lasso problem. The lasso problem seeks a sparse linear combination of the columns of a dictionary to best match a given target vector. This sparse representation has proven useful in a variety of subsequent processing and decision tasks. For a given target vector, dictionary screening quickly identifies a subset of dictionary columns that will receive zero weight in a solution of the corresponding lasso problem. These columns can be removed from the dictionary prior to solving the lasso problem without impacting the optimality of the solution obtained. This has two potential advantages: it reduces the size of the dictionary, allowing the lasso problem to be solved with less resources, and it may speed up obtaining a solution. Using a geometrically intuitive framework, we provide basic insights for understanding useful lasso screening tests and their limitations. We also provide illustrative numerical studies on several datasets.

Music Annotation and Retrieval using Unlabeled Exemplars: Correlation and Sparse Codes

“…Remark on Efficiency: Using LARS Lasso with a Choleskybased implementation, the time complexity of computing is [42], [43], where is the number of iterations needed to reach a local minimum and in general equals to the number of non-zero elements in and . A number of algorithms have been proposed in the literature for better efficiency.…”

“…A number of algorithms have been proposed in the literature for better efficiency. In this letter, we extend the frame-level Lasso screening algorithm proposed by Xiang et al [42], [43] to the clip-level for speeding up, as described below.…”

“…The idea of Lasso screening is to adaptively reduce the size of the dictionary for each input by removing atoms that are unlikely to be non-zero in [43]. It has been proven that we can remove from if the following inequality holds, without affecting the optimal solution of (4) [42]: (6) where and is the possibly negated dictionary atom that leads to .…”

“…Remark on the Difference Between ESC and AESC: Because is not constructed by using theoretically motivated rules [43] such as (6), clip-level Lasso screening can affect the solution of (4). In general, cannot approximate as well as does, and .…”

Stable Safe Screening and Structured Dictionaries for Faster $\ell _{1}$ Regularization

“…For very high-dimensional problems, however, iterative algorithms to solve ℓ 1 minimization problems can become computationally prohibitive, which is why accelerating techniques are still an intense research topic. This paper demonstrates how to combine two such techniques: 1) Fast structured operators [14]- [18] which provide faster matrix-vector products (see Section III-C); 2) Safe screening tests [19]- [24], which safely eliminate unused explanatory variables (see Section II). This paper extends the results in [25] [26].…”