Vector approximate message passing

Abstract

Abstract-The standard linear regression (SLR) problem is to recover a vector x 0 from noisy linear observations y = Ax 0 +w. The approximate message passing (AMP) algorithm recently proposed by Donoho, Maleki, and Montanari is a computationally efficient iterative approach to SLR that has a remarkable property: for large i.i.d. sub-Gaussian matrices A, its periteration behavior is rigorously characterized by a scalar stateevolution whose fixed points, when unique, are Bayes optimal. AMP, however, is fragile in that even small deviations from the i.i.d. sub-Gaussian model can cause the algorithm to diverge. This paper considers a "vector AMP" (VAMP) algorithm and shows that VAMP has a rigorous scalar state-evolution that holds under a much broader class of large random matrices A: those that are right-rotationally invariant. After performing an initial singular value decomposition (SVD) of A, the per-iteration complexity of VAMP can be made similar to that of AMP. In addition, the fixed points of VAMP's state evolution are consistent with the replica prediction of the minimum mean-squared error recently derived by Tulino, Caire, Verdú, and Shamai. The effectiveness and state evolution predictions of VAMP are confirmed in numerical experiments.

Rigorous dynamics of expectation-propagation-based signal recovery from unitarily invariant measurements

“…Gaussian measurement matrices to the case of Haar matrices. This paper presents a constructive proof for the conditional distribution of a Haar matrix, which the correctness of the same result was proved in [24]. The proof strategy of the main theorem is applicable to any MP algorithm for signal recovery from unitarily invariant measurements, such as the AMP, unless the algorithm contains nonlinear processing in the measurement vector y, e.g.…”

“…A similar paper [24] was posted on the arXiv a few months before posting the first version [35] of this paper, of which a short paper will be presented in [1]. Interestingly, the two papers share the common proof strategy based on [12].…”

“…First of all, we consider complex-valued systems, while the posted paper [24] addressed real-valued systems. The main difference is that a probabilistic approach is used in this paper, while a deterministic approach based on pseudo-Lipschitz continuity was considered in [24].…”

“…The main difference is that a probabilistic approach is used in this paper, while a deterministic approach based on pseudo-Lipschitz continuity was considered in [24]. The deterministic approach only provides results averaged over all elements of x.…”

“…As solutions to this convergence issue, since Opper and Winther's pioneering work [18,Appendix D], as well as [19], several algorithms have been proposed on the basis of expectation propagation (EP) [20], expectation consistent (EC) approximations [18], [21], [22], S-transform [23], approximate BP [24], or turbo principle [25]- [28]. The EP-based algorithm [20] is systematically derived from Minka's EP framework [29] by approximating the posterior distribution of x with factorized Gaussian distributions.…”