Analysis of Approximate Message Passing With Non-Separable Denoisers and Markov Random Field Priors

Abstract

Approximate message passing (AMP) is a class of low-complexity, scalable algorithms for solving high-dimensional linear regression tasks where one wishes to recover an unknown signal from noisy, linear measurements. AMP is an iterative algorithm that performs estimation by updating an estimate of the unknown signal at each iteration and the performance of AMP (quantified, for example, by the mean squared error of its estimates) depends on the choice of a "denoiser" function that is used to produce these signal estimates at each iteration.An attractive feature of AMP is that its performance can be tracked by a scalar recursion referred to as state evolution. Previous theoretical analysis of the accuracy of the state evolution predictions has been limited to the use of only separable denoisers or block-separable denoisers, a class of denoisers that underperform when sophisticated dependencies exist between signal entries. Since signals with entrywise dependencies are common in image/video-processing applications, in this work we study the high-dimensional linear regression task when the dependence structure of the input signal is modeled by a Markov random field prior distribution. We provide a rigorous analysis of the performance of AMP, demonstrating the accuracy of the state evolution predictions, when a class of non-separable sliding-window denoisers is applied. Moreover, we provide numerical examples where AMP with sliding-window denoisers can successfully capture local dependencies in images.

Improved bounds for the many-user MAC

“…The behavior of AMP in the high dimensional limit is tracked by the state evolution equations. The convergence of AMP parameters to the state evolution has been proved under various assumptions (see [20][21][22][23][24][25][26]). Furthermore AMP has been successful as a near optimal decoder for sparse superposition codes [25,[27][28][29][30].…”

Compressed Coding, AMP Based Decoding and Analog Spatial Coupling

“…Remark 1: Theorem 2 shows that R AC of the CC scheme with S C can approach C G under two conditions: (a) the matching condition (22) holds and (b) both R C and δ → 0. These conditions require an underlying low-rate FEC code C that meets the matching condition (22). In practice, it is a highly complicated task to design such a low-rate code (see [15]- [17] for details).…”

On the Convergence of Orthogonal/Vector AMP: Long-Memory Message-Passing Strategy

“…where the second equality follows from ( 25), (50), and (55). Thus, we have arrived at (56) with vsuf B→A,t ′ ,t given by (50), instead of (54).…”

“…Since we have defined the covariance messages that are consistent to the state evolution recursions in the large system limit, we have V A→B,τ,τ a.s. → V A→B,τ,τ and V B→A,τ,τ a.s. → V B→A,τ,τ . Thus, (50) and ( 52) reduce to (54) and (58), respectively. This justifies the replacement of v suf B→A,t,t in W t with vsuf B→A,t,t .…”