Low Complexity Regularization of Linear Inverse Problems

Abstract

Inverse problems and regularization theory is a central theme in imaging sciences, statistics and machine learning. The goal is to reconstruct an unknown vector from partial indirect, and possibly noisy, measurements of it. A now standard method for recovering the unknown vector is to solve a convex optimization problem that enforces some prior knowledge about its structure. This chapter delivers a review of recent advances in the field where the regularization prior promotes solutions conforming to some notion of simplicity/low-complexity. These priors encompass as popular examples sparsity and group sparsity (to capture the compressibility of natural signals and images), total variation and analysis sparsity (to promote piecewise regularity), and low-rank (as natural extension of sparsity to matrix-valued data). Our aim is to provide a unified treatment of all these regularizations under a single umbrella, namely the theory of partial smoothness. This framework is very general and accommodates all low-complexity regularizers just mentioned, as well as many others. Partial smoothness turns out to be the canonical way to encode low-dimensional models that can be linear spaces or more general smooth manifolds. This review is intended to serve as a one stop shop toward the understanding of the theoretical properties of the so-regularized solutions. It covers a large spectrum including: (i) recovery guarantees and stability to noise, both in terms of ℓ 2 -stability and model (manifold) identification; (ii) sensitivity analysis to perturbations of the parameters involved (in particular the observations), with applications to unbiased risk estimation ; (iii) convergence properties of the forward-backward proximal splitting scheme, that is particularly well suited to solve the corresponding large-scale regularized optimization problem.

Local Linear Convergence of a Primal-Dual Algorithm for the Augmented Convex Models

“…Therefore, the concept of partly smooth functions seems to provide with a current set of functions that englobes a lot of popular objective models enjoying a powerful calculus and sensitivity theory. In paper [9], where the theory was introduced, a lot of important examples are commented, and in numerous recent papers [6,12,13,20], where the concept of partial smoothness is a key point, more popular objective functions are described as partly smooth functions. For instance, various functions studied in signal processing and machine learning are partly smooth.…”

“…In subsequent papers [12,13,20], the authors extend the definition to convex cases. In this paper, we consider a slightly modified convex version.…”

“…A very active current research subject is the use and development of new algorithms for convex optimization and applications in signal processing, optimization, statistics and machine learning [2,11,14,20,23]. Note that in signal/image processing and compressed sensing some fundamental problems are sparse vector recovery and low-rank matrix recovery, giving rise to minimization problems of suitable objective functions with entries from undetermined linear systems or operators.…”

A Low-Rank Approach to Off-the-Grid Sparse Superresolution

“…Although R λ,ρ obviously differs from the true solution R λ , it is possible to show, following an approach similar to [48], that under some mild nondegeneracy hypothesis on η λ,ρ , and for small enough values of ρ, R λ,ρ is sufficiently close to R λ to allow accurate support reconstruction. In particular, both matrices have the same rank.…”

Sharp, strong and unique minimizers for low complexity robust recovery

“…The structure of the 1 norm plays a crucial role in their result. Sufficient (but not necessary in general) conditions for robust recovery with a linear convergence rate have been proposed for general class of regularizers by solving either (1.4) or (1.5); see [10,14,39,42,36] and references therein. For instance [10] and [36] provide a sufficient condition via the geometric notion of descent cone for robust recovery with linear rate by solving (1.4) (but not (1.5)).…”