Adaptive damping and mean removal for the generalized approximate message passing algorithm

Abstract

The generalized approximate message passing (GAMP) algorithm is an efficient method of MAP or approximate-MMSE estimation of x observed from a noisy version of the transform coefficients z = Ax. In fact, for large zero-mean i.i.d sub-Gaussian A, GAMP is characterized by a state evolution whose fixed points, when unique, are optimal. For generic A, however, GAMP may diverge. In this paper, we propose adaptive-damping and mean-removal strategies that aim to prevent divergence. Numerical results demonstrate significantly enhanced robustness to non-zero-mean, rank-deficient, column-correlated, and ill-conditioned A.

A Two-Stage Method to Test the Robustness of the Generalized Approximate Message Passing Algorithm

“…drawn from N (0, 1). Set the range of γ to [0, 1, 1.8, 1.9, 1.95, 2,3,4,5,6,7,8,9,10,11,12]. Figure 1a shows that GAMP violently diverges at γ = 2, AMP fast diverges at γ = 5.…”

“…They work very well in the zero-mean Gaussian sensing matrix case, but become unstable and divergent in the more general sensing matrix case. For example, although an improvement to the non-zero mean Gaussian sensing matrix case has in a sense been proposed [4], the highly columns-correlated non-Gaussian sensing matrix case is still a problem. We propose a new GAMP-like algorithm, termed Bernoulli-Gaussian Pursuit GAMP (BGP-GAMP), to raise the robustness of standard GAMP algorithm.…”

Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization

“…While the fixed points of the sum-product GAMP algorithm correspond to local minima of the LSL-BFE minimization in (4), the sum-product GAMP algorithm may not always converge (see, e.g., the negative results in [7], [8], [10]). We thus consider an alternative minimization strategy based on a generalization of the classic double-loop method known as the concave convex procedure (CCCP) [16].…”

“…Several recent modifications have been proposed to improve the stability of AMP, including damping [7], sequential updating [9], and adaptive damping [10], some of which have been instrumental in applications such as [11]- [13]. However, while these methods appear to perform empirically well, little has been proven rigorously about their convergence.…”

Intensity-only optical compressive imaging using a multiply scattering material and a double phase retrieval approach

“…measurement matrices [21], however, they do not necessarily converge for generic matrices [22]. There have been some attempts to prevent divergence of AMP-based methods [14,23,24]. In [14], the authors show that a simple change in the main AMP loop may stabilize AMP significantly.…”