Compressed sensing reconstruction using expectation propagation

Abstract

Many interesting problems in fields ranging from telecommunications to computational biology can be formalized in terms of large underdetermined systems of linear equations with additional constraints or regularizers. One of the most studied, the compressed sensing problem, consists in finding the solution with the smallest number of non-zero components of a given system of linear equations y = Fw for known measurement vector y and sensing matrix F. Here, we will address the compressed sensing problem within a Bayesian inference framework where the sparsity constraint is remapped into a singular prior distribution (called Spike-and-Slab or Bernoulli-Gauss). A solution to the problem is attempted through the computation of marginal distributions via expectation propagation, an iterative computational scheme originally developed in statistical physics. We will show that this strategy is more accurate for statistically correlated measurement matrices. For computational strategies based on the Bayesian framework such as variants of belief propagation, this is to be expected, as they implicitly rely on the hypothesis of statistical independence among the entries of the sensing matrix. Perhaps surprisingly, the method outperforms uniformly all the other state-ofthe-art methods in our tests.

Relationship between fitness and heterogeneity in exponentially growing microbial populations

“…Besides, when dealing with the set of dependent and free variables, the statistics of the dependent set is retrieved from those of the free one, according to Eq. 34 as already noticed in [57]. The running time is therefore dominated by one matrix inversion per iteration aimed at computing the covariance matrix of the free fluxes, which scales as O(n 3 ).…”

“…In the following we will first exploit the linear relationship among fluxes to identify the set of dependent and independent fluxes (notice that the Expectation Propagation scheme for this subdivision of the target variables has been already exploited in [57,58] in different contexts). Then, we will derive the two-steps EP update scheme for a general framework in which each experimental flux i ∈ E has its own experimental error and a parameter γ i associated with it (the scheme associated with Eq.…”

“…Among these, we aim at determining the sparsest vector c sp that has the given projection c proj on the polytope and it has the least possible non-zeros components in correspondence to a sub-set a fluxes (we restrict the analysis to the measured fluxes in E, but one can extend the procedure to all fluxes or another relevant sub-set, like internal or irreversible fluxes). To determine c sp we solve the so-called compressed sensing problem (CS) on the original projection of the coefficients using the Expectation Propagation approximation derived in [57]. Within a Bayesian framework we can write the probability of a sparse vector of coefficients, given the projection, using Bayes theorem, as:…”

“…but the computation of any observable from the posterior probability is unfeasible due to the intractability of the normalization (or partition function) of such density. To cope with the marginalization and the estimation of the first moments, we apply the Expectation Propagation approximation to P c sp |c proj where here each spike-and-slab prior is approximated using an univariate Gaussian density, whose parameters are determined using the EP scheme (see [57] for all the details). Surprisingly, even setting ρ → 1, we have found that at least 18 non-zeros components are necessary to fulfill the constraints on the projections, for the full set of coefficients, namely inferred from both the experimental conditions in [28] and [15].…”

Sparse Spectral Unmixing of Hyperspectral Images using Expectation-Propagation

“…EP has already been shown to be an effective method for approximate inference with sparse linear models [32], [35]- [39]. For example, in [32], [37], EP methods were proposed for regression tasks, but without positivity constraints and context information especially for images processing tasks. The work in [36] discusses EP methods in the context of linear regression but with Poisson noise (to photonlimited spectral unmixing).…”