State evolution for approximate message passing with non-separable functions

Berthier, Raphaël; Montanari, Andrea; Nguyen, Phan-Minh

doi:10.1093/imaiai/iay021

Cited by 128 publications

(202 citation statements)

References 47 publications

Supporting

Mentioning

192

Contrasting

Order By: Relevance

“…Our work differs from [18] in the following three aspects: (i ) our work provides finite sample analysis, whereas the result in [18] is asymptotic; (ii ) we adjust the state evolution sequence for the specific class of non-separable sliding-window denoisers to account for the "edge" issue that occurs in the finite sample regime (this point will become clear in later sections); (iii ) we consider the setting where the unknown signal is a realization of an MRF and the expectation in the definition of the state evolution sequence is with respect to (w.r.t.) the signal β, the matrix A, and the noise w, whereas in [18], the signal β is deterministic and unknown, hence the expectation is only w.r.t. the matrix A and the noise w.…”

Section: Introductionmentioning

confidence: 68%

“…We consider 2D/3D MRF priors for the input signal β, and provide performance guarantees for AMP with 2D/3D sliding-window denoisers under some technical conditions. While we were concluding this manuscript, we became aware of recent work of Berthier et al [18]. The authors prove that the loss of the estimates generated by AMP (for a class of loss functions) with general non-separable denoisers converges to the state evolution predictions asymptotically.…”

Section: Introductionmentioning

confidence: 90%

See 1 more Smart Citation

Analysis of Approximate Message Passing With Non-Separable Denoisers and Markov Random Field Priors

Rush

Baron

2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Approximate message passing (AMP) is a class of low-complexity, scalable algorithms for solving high-dimensional linear regression tasks where one wishes to recover an unknown signal from noisy, linear measurements. AMP is an iterative algorithm that performs estimation by updating an estimate of the unknown signal at each iteration and the performance of AMP (quantified, for example, by the mean squared error of its estimates) depends on the choice of a "denoiser" function that is used to produce these signal estimates at each iteration.An attractive feature of AMP is that its performance can be tracked by a scalar recursion referred to as state evolution. Previous theoretical analysis of the accuracy of the state evolution predictions has been limited to the use of only separable denoisers or block-separable denoisers, a class of denoisers that underperform when sophisticated dependencies exist between signal entries. Since signals with entrywise dependencies are common in image/video-processing applications, in this work we study the high-dimensional linear regression task when the dependence structure of the input signal is modeled by a Markov random field prior distribution. We provide a rigorous analysis of the performance of AMP, demonstrating the accuracy of the state evolution predictions, when a class of non-separable sliding-window denoisers is applied. Moreover, we provide numerical examples where AMP with sliding-window denoisers can successfully capture local dependencies in images.

show abstract

Section: Introductionmentioning

confidence: 68%

Section: Introductionmentioning

confidence: 90%

Analysis of Approximate Message Passing With Non-Separable Denoisers and Markov Random Field Priors

Rush

Baron

2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…In the second and third steps, the aim is show that the asymptotic results given in (12) are true. In the second step, we show that our assumptions (A1)-(A4) will allow us to make an appeal to Berthier et al [12,Theorem 14], which provides a relationship between a general SE and the AMP algorithm when the denoiser is non-separable. Finally, in the third step we apply the Berthier et al [12,Theorem 14] result and argue that the SLLN allows us to include the SI.…”

Section: Numerical Resultsmentioning

confidence: 97%

“…Definition 2.1. Pseudo-Lipschitz functions [12]: For k ∈ N >0 and any n, m ∈ N >0 , a function φ : R n → R m is pseudo-Lipschitz of order k, or PL(k), if there exists a constant L, referred to as the pseudo-Lipschitz constant of φ, such that for x, y ∈ R n ,…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

An Analysis of State Evolution for Approximate Message Passing with Side Information

Liu

Rush

Baron

2019

2019 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

A common goal in many research areas is to reconstruct an unknown signal x from noisy linear measurements. Approximate message passing (AMP) is a class of low-complexity algorithms that can be used for efficiently solving such high-dimensional regression tasks. Often, it is the case that side information (SI) is available during reconstruction. For this reason, a novel algorithmic framework that incorporates SI into AMP, referred to as approximate message passing with side information (AMP-SI), has been recently introduced. In this work, we provide rigorous performance guarantees for AMP-SI when there are statistical dependencies between the signal and SI pairs and the entries of the measurement matrix are independent and identically distributed (i.i.d.) Gaussian. The AMP-SI performance is shown to be provably tracked by a scalar iteration referred to as state evolution (SE). Moreover, we provide numerical examples that demonstrate empirically that the SE can predict the AMP-SI mean square error accurately. * A subset of this work has been presented at

show abstract

Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models

2021

View full text Add to dashboard Cite

We study the problem of recovering an unknown signal $${\varvec{x}}$$ x given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator $$\hat{\varvec{x}}^\mathrm{L}$$ x ^ L and a spectral estimator $$\hat{\varvec{x}}^\mathrm{s}$$ x ^ s . The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine $$\hat{\varvec{x}}^\mathrm{L}$$ x ^ L and $$\hat{\varvec{x}}^\mathrm{s}$$ x ^ s . At the heart of our analysis is the exact characterization of the empirical joint distribution of $$({\varvec{x}}, \hat{\varvec{x}}^\mathrm{L}, \hat{\varvec{x}}^\mathrm{s})$$ ( x , x ^ L , x ^ s ) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of $$\hat{\varvec{x}}^\mathrm{L}$$ x ^ L and $$\hat{\varvec{x}}^\mathrm{s}$$ x ^ s , given the limiting distribution of the signal $${\varvec{x}}$$ x . When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form $$\theta \hat{\varvec{x}}^\mathrm{L}+\hat{\varvec{x}}^\mathrm{s}$$ θ x ^ L + x ^ s and we derive the optimal combination coefficient. In order to establish the limiting distribution of $$({\varvec{x}}, \hat{\varvec{x}}^\mathrm{L}, \hat{\varvec{x}}^\mathrm{s})$$ ( x , x ^ L , x ^ s ) , we design and analyze an approximate message passing algorithm whose iterates give $$\hat{\varvec{x}}^\mathrm{L}$$ x ^ L and approach $$\hat{\varvec{x}}^\mathrm{s}$$ x ^ s . Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.

show abstract

State evolution for approximate message passing with non-separable functions

Cited by 128 publications

References 47 publications

Analysis of Approximate Message Passing With Non-Separable Denoisers and Markov Random Field Priors

Analysis of Approximate Message Passing With Non-Separable Denoisers and Markov Random Field Priors

An Analysis of State Evolution for Approximate Message Passing with Side Information

Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models

Contact Info

Product

Resources

About