A Unifying Tutorial on Approximate Message Passing

Feng, Oliver Y.; Venkataramanan, Ramji; Rush, Cynthia; Samworth, Richard J.

doi:10.1561/2200000092

Cited by 25 publications

(14 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another important class of algorithms uses approximate message passing (AMP), which is a powerful technique that has been utilized extensively in high-dimensional statistics [38]. Variants of AMP have successfully been devised with theoretical guarantees in several inverse problems with generative priors, including linear forward models [86], [39], spiked matrix recovery [6], and phase retrieval [5].…”

Section: E Further Developmentsmentioning

confidence: 99%

“…The reformulation of τ ISTA iterations onto a τ -layer neural network with parameters that can be further tuned, as described in (38), (39), and (40), is referred to as LISTA. It is often referred to as a model-based learning method, because the network architecture is specifically defined according to a particular measurement model (the linear model), optimization procedure (Lasso), and iterative solver (ISTA).…”

Section: B the Unfolding Principle: Listamentioning

confidence: 99%

See 1 more Smart Citation

Theoretical Perspectives on Deep Learning Methods in Inverse Problems

Scarlett¹,

Heckel²,

Rodrigues³

et al. 2022

Preprint

View full text Add to dashboard Cite

In recent years, there have been significant advances in the use of deep learning methods in inverse problems such as denoising, compressive sensing, inpainting, and super-resolution. While this line of works has predominantly been driven by practical algorithms and experiments, it has also given rise to a variety of intriguing theoretical problems. In this paper, we survey some of the prominent theoretical developments in this line of works, focusing in particular on generative priors, untrained neural network priors, and unfolding algorithms. In addition to summarizing existing results in these topics, we highlight several ongoing challenges and open problems.

show abstract

Section: E Further Developmentsmentioning

confidence: 99%

Section: B the Unfolding Principle: Listamentioning

confidence: 99%

Theoretical Perspectives on Deep Learning Methods in Inverse Problems

Scarlett¹,

Heckel²,

Rodrigues³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To save space and to avoid introducing further notation, we only state an informal theorem of this result here, and we refer to the statement of the GAMP algorithm in [17, Section 4 Equation (55)] and its corresponding state evolution in [17, Section 4 Equation (57)-( 58)]. [17,Section 4 Equation (55)] and its state evolution in [17,Section 4 Equation ( 57)-( 58)], the asymptotic equivalence of its iterates and its state evolution (see for example for an asymptotic statement in [17,Section 4 Theorem 4.2]) can be upgraded to exponentially fast concentration with rates of…”

Section: Theorem 1 [Vamp and Gvamp Concentration]mentioning

confidence: 99%

“…AMP-style algorithms can accommodate a range of estimation procedures for the models in (1.1)-(1.2), including maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. See [17] for a tutorial on AMP.…”

Section: Introductionmentioning

confidence: 99%

Exponentially Fast Concentration of Vector Approximate Message Passing to its State Evolution

Cademartori

Rush

2020

2020 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

Approximate Message Passing (AMP) algorithms are a class of iterative procedures for computationally-efficient estimation in high-dimensional inference and estimation tasks. Due to the presence of an 'Onsager' correction term in its iterates, for N × M design matrices A with i.i.d. Gaussian entries, the asymptotic distribution of the estimate at any iteration of the algorithm can be exactly characterized in the large system limit as M/N → δ ∈ (0, ∞) via a scalar recursion referred to as state evolution. In this paper, we show that appropriate functionals of the iterates, in fact, concentrate around their limiting values predicted by these asymptotic distributions with rates exponentially fast in N for a large class of AMP-style algorithms, including those that are used when high-dimensional generalized linear regression models are assumed to be the data-generating process, like the generalized AMP algorithm, or those that are used when the measurement matrix is assumed to be right rotationally invariant instead of i.i.d. Gaussian, like vector AMP and generalized vector AMP. In practice, these more general AMP algorithms have many applications, for example in in communications or imaging, and this work provides the first study of finite sample behavior of such algorithms. c 2k (k!) κ ′ with κ, κ ′ and c, c ′ being universal constants not depending on ǫ or N . While we do not specify κ, κ ′ explicitly in this informal statement, these will be small constant values (like κ = 16 and κ ′ = 22 from Theorem 1).

show abstract

“…In [AHMN23], it is shown that in the GOE case, the asymptotic behavior of µ x‹ , where u ‹ is now the LCP solution, can be evaluated with the help of an Approximate Message Passing (AMP) technique. Such techniques have recently aroused an intense and growing research effort in the fields of statistical physics, communication theory, or statistical Machine Learning [FVRS22]. In a word, given a function h : R ˆR ˆN Ñ R and a random symmetric n ˆn so-called measurement matrix W , a standard AMP algorithm is an iterative algorithm of the form xt`1 " W hpx t , η, tq `a "correction" term, where η " rη i s n i"1 P R n is a parameter vector, and where hpx t , η, tq " " hpx t i , η i , tq…”

Section: Introductionmentioning

confidence: 99%

Approximate Message Passing for sparse matrices with application to the equilibria of large ecological Lotka-Volterra systems

Hachem¹

2023

Preprint

View full text Add to dashboard Cite

This paper is divided into two parts. The first part is devoted to the study of a class of Approximate Message Passing (AMP) algorithms which are widely used in the fields of statistical physics, machine learning, or communication theory. The AMP algorithms studied in this part are those where the measurement matrix has independent elements, up to the symmetry constraint when this matrix is symmetric, with a variance profile that can be sparse. The AMP problem is solved by adapting the approach of Bayati, Lelarge, and Montanari (2015) to this matrix model. The Lotka-Volterra (LV) model is the standard model for studying the dynamical behavior of large dimensional ecological food chains. The second part of this paper is focused on the study of the statistical distribution of the globally stable equilibrium vector of a LV system in the situation where the random symmetric interaction matrix among the living species is sparse, and in the regime of large dimensions. This equilibirium vector is the solution of a Linear Complementarity Problem, which distribution is shown to be characterized through the AMP approach developed in the first part. In the large dimensional regime, this distribution is close to a mixture of a large number of truncated Gaussians.

show abstract

A Unifying Tutorial on Approximate Message Passing

Cited by 25 publications

References 0 publications

Theoretical Perspectives on Deep Learning Methods in Inverse Problems

Theoretical Perspectives on Deep Learning Methods in Inverse Problems

Exponentially Fast Concentration of Vector Approximate Message Passing to its State Evolution

Approximate Message Passing for sparse matrices with application to the equilibria of large ecological Lotka-Volterra systems

Contact Info

Product

Resources

About