An efficient and globally convergent algorithm for $\ell_{p,q} - \ell_r$ model in group sparse optimization

Abstract

Group sparsity combines the underlying sparsity and group structure of the data in problems. We develop a proximally linearized algorithm InISSAPL for the non-Lipschitz group sparse p,qr optimization problem. The algorithm gives a unified framework for all the parameters p ≥ 1, 0 < q < 1, 1 ≤ r ≤ ∞, which is applicable to different kinds of measurement noise. In particular, it includes the addition of the non-smooth 1,q regularization term and the non-smooth 1 / ∞ fidelity term as special cases. It allows an inexact inner loop accessible to the implementation of scaled ADMM, and still has global convergence. The algorithm is efficient and fast with computation only on the shrinking group support set. Many numerical experiments are presented for the algorithm with diversity of parameters p, q, r. The comparisons show that our algorithm is superior to others in the existing works.Laplace noise or heavy-tailed noise such as impulsive noise, the 1 fidelity term (r = 1) is a good choice. For the noise by uniform distribution or quantization error, the ∞ fidelity term (r = ∞) suits.There are many references to study the sparse optimization problem without group structure in it, i.e. the non-group model in which the number of groups g equals N . Then the p,q term in (1.1) is degenerated to q (0 < q < 1) regularization one. One class of methods is smoothing approximate methods [5,[12][13][14]24]. By a smoothing function ϕ(x, θ), the non-Lipschitz property of the objective function can be removed. The second class of methods is general iterative shrinkage-thresholding algorithms (GISA) for q -2 problem [9,34,40]. GISA was inspired by the great success of soft thresholding and iterative shrinkage-thresholding algorithms (ISTA) [3,16] for convex 1 -2 problem. The third class of methods is the iterative reweighted minimization methods for q -2 minimization problem; see, e.g. [10,15,23,27]. Actually reweighted methods reformulate the original non-Lipschitz q -2 to Lipschitz ones by a de-singularizing parameter. Very recently, [25,39] developed methods by successively shrinking the support of the variables to overcome non-Lipschitz property, in which [25] considered the non-group case with r = ∞ and [39] focused on the image restoration with r = 2. To the best of our knowledge, we note that most of the references considered only r = 2 in these methods.For the group sparse optimization problem (1.1), most algorithms were proposed only in the case of r = 2 as well. Hu et al. [21] investigated this problem via p,q regularization, others developed algorithms for 2,1 regularized least squares, e.g. group Lasso [8,17,38]. As noted before, it is important and necessary to develop algorithm for general 1 ≤ r ≤ ∞. This will bring the difficulty to universally handle the noise parameter r with regularized parameters p, q in the group structure. In addition, the regularization term p,q with parameters p ≥ 1, 0 < q < 1 in the objective function E in (1.1) leads to a non-convex, non-Lipschitz optimization problem. This non-smoothness becom...

A globally convergent algorithm for a class of gradient compounded non-Lipschitz models applied to non-additive noise removal

“…Theorem 3.1 naturally motivates an iterative support shrinkage procedure, like [30,44,45,47] for different signal and image recovery problems. Given u k , it computes u k+1 by solving the following constrained problem:…”

A General Non-Lipschitz Joint Regularized Model for Multi-Channel/Modality Image Reconstruction

“…We mention that this support shrinking strategy was recently derived and used with the iteratively reweighted ℓ 1 [10,24] or least squares [18,34] algorithmic structure, for different signal and image processing problems [23,38,53,55,57,60].…”

“…In [55,57], the authors proposed a linearized proximal technique with an exact subsolver for each outer iteration. Inspired by [3,38,53,58,60] for different minimization problems, we propose the following inexact iterative support shrinking algorithm with linearization and projection, which is expected to solve the inner subproblem inexactly and terminate it by a subgradient error criterion. Please note that we recall the indicator function of a set U as…”

“…Although there are several recent good advances [31,37,51], the gradient operator ∇ in 2D image restoration problems does not fit their assumptions on the linear operator. Very recently, an iterative support shrinking strategy has been derived and used with the iteratively reweighted ℓ 1 [10,24] or least squares [18,34] algorithmic structure, for various signal and image restoration problems [23,38,49,53,55,57]. In particular, the parallel [55,57] considered the gradient compounded non-Lipschitz variational model (1.3) in the discrete setting with q = 1,2 for the single channel case, i.e., M = 1, and constructed the iterative support shrinking algorithm with proximal linearization (ISS-APL) with convergence results established by Kurdyka-Łojasiewicz (KL) property [1][2][3].…”

An Efficient Semismooth Newton Method for Adaptive Sparse Signal Recovery Problems

“…The data fidelity with form Ax−b 1 has also been evidently shown to be more robust when the noises are not normal but heavy-tailed or heterogeneous, see e.g., ; Lu (2014); Wang (2013); Xiu et al (2018). Besides, the data fidelity with form Ax − b ∞ is also known to be very suitable for dealing with the uniformly distributed noise and quantization error, see e.g., Wen et al (2018); Xue et al (2019); Zhang and Wei (2015). Therefore, a more natural question is whether or not one can design a more flexible and robust reconstruction model as well as an efficient algorithm which is capable of dealing with all the three types of noise mentioned above?…”