A Survey on Nonconvex Regularization-Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning

Abstract

In the past decade, sparse and low-rank recovery have drawn much attention in many areas such as signal/image processing, statistics, bioinformatics and machine learning. To achieve sparsity and/or low-rankness inducing, the 1 norm and nuclear norm are of the most popular regularization penalties due to their convexity. While the 1 and nuclear norm are convenient as the related convex optimization problems are usually tractable, it has been shown in many applications that a nonconvex penalty can yield significantly better performance. In recent, nonconvex regularization based sparse and low-rank recovery is of considerable interest and it in fact is a main driver of the recent progress in nonconvex and nonsmooth optimization. This paper gives an overview of this topic in various fields in signal processing, statistics and machine learning, including compressive sensing (CS), sparse regression and variable selection, sparse signals separation, sparse principal component analysis (PCA), large covariance and inverse covariance matrices estimation, matrix completion, and robust PCA. We present recent developments of nonconvex regularization based sparse and low-rank recovery in these fields, addressing the issues of penalty selection, applications and the convergence of nonconvex algorithms. Code is available at https://github.com/FWen/ncreg.git.

Negentropy-Based Sparsity-Promoting Reconstruction with Fast Iterative Solution from Noisy Measurements

“… regularization, a nonconvex and non-smooth optimization problem, has been proven to solve more accurate sparse solutions than regularization as a relaxation approach in [ 27 , 28 , 29 ]. Weighted regularization is an approximate representation of regularization [ 30 , 31 ].…”

New Robust Principal Component Analysis for Joint Image Alignment and Recovery via Affine Transformations, Frobenius and L2,1 Norms

“…In contrast, the L 2,1 norm is more stable and has the ability to better preserve the spatial information than the L 1 regularizer, as demonstrated in [13,47]. Additionally, the L 2,1 norm is superior to the nonconvex norms when the signals are not sparse or when the matrix is not strictly low rank [48,49]. e overall problem can thus be posted as an optimization problem given by minimize L,S,Δτ…”

Efficient Algorithms for Global Outlier Removal in Large-scale Structure-from-Motion

“…The dual problem (12) can be solved by well-developed LP solvers. The algorithm is summarized as follows.…”

“…or q (0 < q < 1) norm can usually yield a sparser solution than the 1 norm[12]. Empirical results have shown that the 1 minimization (10) is likely to remove inliers in some conditions[6],[24].…”