Upscaling Vector Approximate Message Passing

Abstract

Recently proposed Vector Approximate Message Passing (VAMP) demonstrates a great reconstruction potential at solving compressed sensing related linear inverse problems. VAMP provides high per-iteration improvement, can utilize powerful denoisers like BM3D, has rigorously defined dynamics and is able to recover signals sampled by highly undersampled and ill-conditioned linear operators. Yet, its applicability is limited to relatively small problem sizes due to necessity to compute the expensive LMMSE estimator at each iteration. In this work we consider the problem of upscaling VAMP by utilizing Conjugate Gradient (CG) to approximate the intractable LMMSE estimator and propose a CG-VAMP algorithm that can efficiently recover large-scale data. We derive evolution models of certain key parameters of CG-VAMP and use the theoretical results to develop fast and practical tools for correcting, tuning and accelerating the CG algorithm within CG-VAMP to preserve all the main advantages of VAMP, while maintaining reasonable and controllable computational cost of the algorithm.

Divergence Estimation in Message Passing algorithms

“…Thus, we have a well-defined γ t t = γ t and γ τ = 0, τ < t for CG-VAMP. Lastly, the linear update f L with F t (Z t ) in WS-CG-VAMP also fits into the model (72) as shown in Theorem 3 in [20]. Thus, the result (75) holds for MF-OAMP, VAMP, CG-VAMP and WS-CG-VAMP.…”

“…The first group of algorithms, which we refer as OAMPbased algorithms, constructs f D and f L such that their output errors are asymptotically orthogonal to their input errors [10]. This group of algorithms includes Match Filter OAMP (MF-OAMP) [10], [27], VAMP [9], [24], CG-VAMP [25], [19], [20], WS-CG-VAMP [19] and others. Here the structure of the function…”

“…We consider the large scale compressed sensing scenario M < N with a subsampling factor δ = M N = O(1). While there are many first-order iterative methods for recovering x from the set of measurement (1) including [11], [26], [3], [7] and many others, in this work we focus on the family of Scalable Message Passing (SMP) algorithms that includes Approximate Message Passing (AMP) [8], Orthogonal AMP (OAMP) [10], Vector AMP (VAMP) [16], Conjugate Gradient VAMP (CG-VAMP) [19], [25], [20], Warm-Started CG-VAMP (WS-CG-VAMP) [19], Convolutional AMP (CAMP) [23] and others. When the measurement operator A comes from a certain family of random matrices, which may be different for each example of SMP, these algorithms demonstrate high periteration improvement and stable and predictable dynamics.…”

Compressed Sensing With Upscaled Vector Approximate Message Passing

“…Moreover, if the approximation produced by CG is sufficiently bad, then v t,i A→B might even increase with respect to v t−1 A→B . Based on numerical experiments in the current and the previous work [32], we have observed that the number of the inner-loop iterations sufficient to achieve a certain reduction of v t,i A→B per-outer-loop iteration significantly changes with t and therefore motivates the benefit in the adaptive choice of i[t] at each t.…”

“…Practical estimation of v t,i A→B in CG-VAMP Next, we propose an asymptotically consistent estimator for the variance v t,i A→B that can be naturally implemented within CG and has negligible computational and memory costs. For this, we expand the result (32) and use the identity (18) to obtain v t,i A→B a.s.…”