Hybrid Approximate Message Passing

Abstract

Abstract-Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing small, linearizable couplings of variables. AMP approximations based on the Central Limit Theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance-complexity tradeoff can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.

Sketched clustering via hybrid approximate message passing

“…estimand c and separable likelihood p(y|z) = M m=1 p(y m |z m ). Thus, Hybrid GAMP (HyGAMP) [18] was developed to tackle problems with a structured prior and/or likelihood. HyGAMP could be applied to the compressive learning problem described in Section II-A, but it would require computing and inverting O(N+M ) covariance matrices of dimension K at each iteration.…”

Hybrid Vector Perturbation Precoding: The Blessing of Approximate Message Passing

“…Fortunately, it is known that a basis is short and nearly orthogonal after lattice reduction, which means its column-wise dependency is small. Moreover, a reduced basis in VP often has "small" entries (in the sense of [35]) such that the approximations in AMP are valid. We further justified the two arguments above in Appendix A.…”

Sketched Clustering via Hybrid Approximate Message Passing

“…estimand c and separable likelihood p(y|z) = M m=1 p(y m |z m ). Thus, Hybrid GAMP (HyGAMP) [13] was developed to tackle problems with a structured prior and/or likelihood. HyGAMP could be applied to (10)- (11), but it requires computing and inverting O(N +M ) covariance matrices of dimension K at each iteration.…”

“…Require: Measurements y, matrix W with ∥ W ∥ 2 F = M , pdfs p c|r and p z|y,p from (12)- (13), initializations C = E{C}, q c n = diag(cov{cn}) Ensure: S ← 0.…”