Sparse Regularization via Convex Analysis

Abstract

Sparse approximate solutions to linear equations are classically obtained via L1 norm regularized least squares, but this method often underestimates the true solution. As an alternative to the L1 norm, this paper proposes a class of non-convex penalty functions that maintain the convexity of the least squares cost function to be minimized, and avoids the systematic underestimation characteristic of L1 norm regularization. The proposed penalty function is a multivariate generalization of the minimax-concave (MC) penalty. It is defined in terms of a new multivariate generalization of the Huber function, which in turn is defined via infimal convolution. The proposed sparse-regularized least squares cost function can be minimized by proximal algorithms comprising simple computations.The ℓ 1 norm is classically used as a regularizer here, since among convex regularizers it induces sparsity most effectively [9]. But this formulation tends to underestimate high-amplitude components of x ∈ R N . Non-convex sparsity-inducing regularizers are also widely used (leading to more accurate estimation of high-amplitude components), but then the cost function is generally non-convex and has extraneous suboptimal local minimizers [43].

Linearly involved generalized Moreau enhanced models and their proximal splitting algorithm under overall convexity condition

“…For the whole shape of the graph of ( · 1 ) B , see the graphs in [56,Figs. 3,8,and 9] of the GMC penalty.…”

“…The GMC penalty function is a parameterized multidimensional extension of the minimax concave (MC) penalty function [71] (see also [4,28]) 2 . It is known that (i) the GMC penalty function ( · 1 ) B is nonconvex except for ( · 1 ) Oq×n = · 1 (see Remark 3(ii)); (ii) for any A ∈ R m×n , ( · 1 ) B can maintain the overall 1 We use the notation ( · 1 ) B in place of its original notation Ψ B used in [56] for the GMC penalty because the GMC penalty in [56] was introduced as a nonconvex alternative to · 1 with B ∈ R q×n . In Definition 1 of the present paper, we will use Ψ B in much wider sense to denote a nonconvex alternative to a general proximable convex function Ψ defined on finite dimensional real Hilbert space.…”

“…an iterative algorithm was presented by Selesnick [56] (see Appendix A) but only for a special case satisfying B * B = (θ/µ)A * A (0 ≤ θ ≤ 1). Despite its great potential of the GMC penalty, so far the applicability of the algorithm in [56] is very limited. For example, it is not applicable directly to most scenarios in Example 1(c) and (d).…”

“…(b) (On Q2) In [56] specially for (X , Z, Ψ, L) = (R n , R n , · 1 , Id), a sufficient condition is found for B and µ to ensure the convexity of J ( · 1)B •Id . We will see in Remark 3 that this sufficient condition is indeed a necessary and sufficient condition to ensure the convexity of J ( · 1)B •Id .…”

Non-convex Optimization via Strongly Convex Majorization-minimization

“…The algorithm converges to a stationary point of cost function when the surregators are strongly convex. For the class of our optimization problems, the proposed penalizer of the cost function is the difference of 1 -norm and the Moreau envelope of a convex function, and it is a generalization of GMC non-separable penalty function previously introduced by Ivan Selesnick in [11].…”