<formula formulatype="inline"><tex Notation="TeX">$L_{1/2}$</tex> </formula> Regularization: Convergence of Iterative Half Thresholding Algorithm

Abstract

In recent studies on sparse modeling, the nonconvex regularization approaches (particularly, regularization with ) have been demonstrated to possess capability of gaining much benefit in sparsity-inducing and efficiency. As compared with the convex regularization approaches (say, regularization), however, the convergence issue of the corresponding algorithms are more difficult to tackle. In this paper, we deal with this difficult issue for a specific but typical nonconvex regularization scheme, the regularization, which has been successfully used to many applications. More specifically, we study the convergence of the iterative half thresholding algorithm (the half algorithm for short), one of the most efficient and important algorithms for solution to the regularization. As the main result, we show that under certain conditions, the half algorithm converges to a local minimizer of the regularization, with an eventually linear convergence rate. The established result provides a theoretical guarantee for a wide range of applications of the half algorithm. We provide also a set of simulations to support the correctness of theoretical assertions and compare the time efficiency of the half algorithm with other known typical algorithms for regularization like the iteratively reweighted least squares (IRLS) algorithm and the iteratively reweighted minimization (IRL1) algorithm.Index Terms-Convergence, iterative half thresholding algorithm, regularization, nonconvex regularization.

Sparse regularized optimization via Gauss-Seidel iterative thresholding

“…一些普通的算法, 例如, 文献 [6,8,13,15,[23][24][25] 以及它们的参考文献, 都不是通过使用 Gauss-Seidel 方法来更新迭代的. 文献 [15] 提出了迭代跳跃 阈值 (iterative jumping thresholding, IJT) 算法, [10] 、软阈值 [11] 和半阈值 [12,14]…”

Sparse Adaptive Iteratively-Weighted Thresholding Algorithm (SAITA) for L p -Regularization Using the Multiple Sub-Dictionary Representation

“…In this section, we first introduce the iteratively thresholding representation theory for the conventional $L_p,\left(p\in \left\{1/2,2/3\right\}\right)$ algorithm according to the existing series of algorithms in [25,34]. Then, we deduce the SAITA-$L_p$ algorithm combined with the proposed weighted scheme ${\lambda }_{d,p}$.…”

SPARSE REPRESENTATION LEARNING OF DATA BY AUTOENCODERS WITH <em>L</em>_1/2 REGULARIZATION

“…The Lasso method and its variants [22][23][24] have made L 1 regularization become a popular data analysis algorithm. Later on, an L 1/2 regularization method was proposed in [25][26][27], which has some promising properties. Fig.…”