Expectation-Maximization Gaussian-Mixture Approximate Message Passing

Abstract

When recovering a sparse signal from noisy compressive linear measurements, the distribution of the signal's non-zero coefficients can have a profound effect on recovery mean-squared error (MSE). If this distribution was apriori known, then one could use computationally efficient approximate message passing (AMP) techniques for nearly minimum MSE (MMSE) recovery. In practice, though, the distribution is unknown, motivating the use of robust algorithms like LASSO---which is nearly minimax optimal---at the cost of significantly larger MSE for non-least-favorable distributions. As an alternative, we propose an empirical-Bayesian technique that simultaneously learns the signal distribution while MMSE-recovering the signal---according to the learned distribution---using AMP. In particular, we model the non-zero distribution as a Gaussian mixture, and learn its parameters through expectation maximization, using AMP to implement the expectation step. Numerical experiments on a wide range of signal classes confirm the state-of-the-art performance of our approach, in both reconstruction error and runtime, in the high-dimensional regime, for most (but not all) sensing operators

Bilinear Generalized Approximate Message Passing—Part I: Derivation

“…In this section, we outline a methodology that takes a given set of BiG-AMP parameterized priors and tunes the parameter vector using an expectation-maximization (EM) [37] based approach, with the goal of maximizing the likelihood, i.e., finding . The approach presented here can be considered as a generalization of the GAMP-based work [38] to BiG-AMP.…”

“…In addition, we propose an adaptive damping [36] mechanism, an expectation-maximization (EM)-based [37] method of tuning the parameters of , , and (in case they are unknown), and methods to select the rank (in case it is unknown). In the case that , , and/or are completely unknown, they can be modeled as Gaussian-mixtures with mean/variance/weight parameters learned via EM [38]. In Part II [1], we detail the application of BiG-AMP to matrix completion, robust PCA, and dictionary learning, and present the results of an extensive numerical investigation into the performance of BiG-AMP in each application.…”

Multipath Mitigation in Global Navigation Satellite Systems Using a Bayesian Hierarchical Model With Bernoulli Laplacian Priors

“…In order to promote sparsity, we assign Bernoulli-Laplace priorr to these vectors. Note that this kind of prior has been used successfully in different applications [5,6,7]. Based on these works, the following probability density function (pdf) is chosen as prior for m i,k…”

A Robust Non-Gaussian Data Assimilation Method for Highly Non-Linear Models

“…One example of M could be a model that mimics the ocean and/or the atmosphere dynamics. In traditional DA settings, prior errors are described by Gaussian distributions, where, for the k-th mixture component, x b k ∈ R n×1 is the mean, B k ∈ R n×n is the background error covariance, and α b k is the prior weight, for 1 ≤ k ≤ K. These weights can be estimated, for instance, using the Expectation Maximization algorithm [6][7][8]. In sequential methods, it is common to assume Gaussian errors over observations y ∈ R m×1 , y ∼ N (H (x) , R)…”