Provable Compressed Sensing with Generative Priors via Langevin Dynamics

Abstract

Deep generative models have emerged as a powerful class of priors for signals in various inverse problems such as compressed sensing, phase retrieval and super-resolution. Here, we assume an unknown signal to lie in the range of some pre-trained generative model. A popular approach for signal recovery is via gradient descent in the low-dimensional latent space. While gradient descent has achieved good empirical performance, its theoretical behavior is not well understood. In this paper, we introduce the use of stochastic gradient Langevin dynamics (SGLD) for compressed sensing with a generative prior. Under mild assumptions on the generative model, we prove the convergence of SGLD to the true signal. We also demonstrate competitive empirical performance to standard gradient descent.

Theoretical Perspectives on Deep Learning Methods in Inverse Problems

“…Two further works gave guarantees that require neither random weights nor exact projections, but instead adopt further deterministic assumptions on G, roughly amounting to certain forms of smoothness. Specifically, [46] studied an algorithm based on Alternating Direction Method-of-Multipliers (ADMM), and [100] studied an algorithm based on Langevin dynamics. Algorithms based on Langevin dynamics have also been explored in several other works on inverse problems and beyond, e.g., see [67] for its use in posterior sampling with general probabilistic priors, and [106] for a general nonasymptotic analysis under non-convex objectives.…”

Theoretical Perspectives on Deep Learning Methods in Inverse Problems

“…Two further works gave guarantees that require neither random weights nor exact projections, but instead adopt further deterministic assumptions on G, roughly amounting to certain forms of smoothness. Specifically, [42] studied an algorithm based on Alternating Direction Method-of-Multipliers (ADMM), and [93] studied an algorithm based on Langevin dynamics.…”