A Unified Primal Dual Active Set Algorithm for Nonconvex Sparse Recovery

Abstract

In this paper, we consider the problem of recovering a sparse signal based on penalized least squares formulations. We develop a novel algorithm of primal-dual active set type for a class of nonconvex sparsity-promoting penalties, including 0 , bridge, smoothly clipped absolute deviation, capped 1 and minimax concavity penalty. First we establish the existence of a global minimizer for the related optimization problems. Then we derive a novel necessary optimality condition for the global minimizer using the associated thresholding operator. The solutions to the optimality system are coordinate-wise minimizers, and under minor conditions, they are also local minimizers. Upon introducing the dual variable, the active set can be determined using the primal and dual variables together. Further, this relation lends itself to an iterative algorithm of active set type which at each step involves first updating the primal variable only on the active set and then updating the dual variable explicitly. When combined with a continuation strategy on the regularization parameter, the primal dual active set method is shown to converge globally to the underlying regression target under certain regularity conditions. Extensive numerical experiments with both simulated and real data demonstrate its superior performance in efficiency and accuracy compared with the existing sparse recovery methods.

A Semismooth Newton Algorithm for High-Dimensional Nonconvex Sparse Learning

“…To select an "optimal" value of the tuning parameter λ, a high-dimensional Bayesian information criterion (HBIC) [33], [34] can be utilized. In this paper, we also propose to use a novel voting criterion (VC) [32], [35] for choosing the optimal value of λ. Assume we run the SSN algorithm and obtain a solution path until, for example, β(λ t ) 0 > ⌊n/ log(p)⌋ for some t, say t = W (W ≤ M ).…”

Sparse signal reconstruction via the approximations of <inline-formula><tex-math id="M1">\begin{document}$ \ell_{0} $\end{document}</tex-math></inline-formula> quasinorm

“…Clearly, the optimal solution of problem (1) is the sparsest solution to the linear system Ay = b. Furthermore, there exist a variety of algorithms which have been proposed to solve problem (1), including orthogonal matching pursuit (OMP) [42], compressive sampling matching pursuit (CoSaMP) [35] and others [23,24,25,33,37,38].…”

“…The key feature of the cardinality minimization problem (1) is the 0 quasinorm. There are already many methods to deal with the 0 quasinorm [5,9,29,21,23,33,37,46]. From a convex analysis point of view, a natural methodology is to minimize the convex envelope of y 0 .…”