Approximate inference techniques with expectation constraints

Heskes, Tom; Opper, Manfred; Wiegerinck, Wim; Winther, Ole; Zoeter, Onno

doi:10.1088/1742-5468/2005/11/p11015

Cited by 42 publications

(42 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Of course the choice of decomposition should be guided not only be tractability but also by quality of the approximation. We expect from central limit theorem (CLT) arguments that the EC approximation with this decomposition will become better the higher the number of sources with a "homogeneous" connectivity of the mixing matrix [15,16,5]. Empirically we observe that that the EC approximation is almost always more precise than the variational approximation even for quite small systems where we cannot really rely on the CLT argument.…”

Section: Expectation Consistentmentioning

confidence: 92%

See 1 more Smart Citation

Flexible and efficient implementations of Bayesian independent component analysis

Winther

Petersen

2007

Neurocomputing

Self Cite

View full text Add to dashboard Cite

In this paper we present an empirical Bayes method for flexible and efficient Independent Component Analysis (ICA). The method is flexible with respect to choice of source prior, dimensionality and positivity of the mixing matrix, and structure of the noise covariance matrix. The efficiency is ensured using parameter optimizers which are more advanced than the expectation maximization (EM) algorithm, but still easy to implement. These optimizers are the overrelaxed adaptive EM algorithm and the easy gradient recipe. The required expectations over the source posterior are estimated with accurate mean field methods: variational and the expectation consistent framework.We demonstrate the usefulness of the approach with the publicly available Matlab toolbox icaMF. Flexible and Efficient Implementations of Bayesian Independent Component Analysis AbstractIn this paper we present an empirical Bayes method for flexible and efficient Independent Component Analysis (ICA). The method is flexible with respect to choice of source prior, dimensionality and constraints of the mixing matrix (positivity), and structure of the noise covariance matrix. The efficiency is ensured using parameter optimizers which are more advanced than the expectation maximization (EM) algorithm, but still easy to implement. These optimizers are the overrelaxed adaptive EM algorithm and the easy gradient recipe. The required expectations over the source posterior are estimated with mean field methods: variational and the expectation consistent (EC) framework. We describe the derivation of the EC framework for ICA in detail and give empirical results demonstrating the improved performance. We demonstrate the usefulness of the approach with the publicly available Matlab toolbox icaMF.

show abstract

Section: Expectation Consistentmentioning

confidence: 92%

“…EC and EP are closely related to the adaptive TAP framework [15,16,6,5]. In fact these non-linear iterative methods share fixed points.…”

Section: Introductionmentioning

confidence: 99%

Flexible and efficient implementations of Bayesian independent component analysis

Winther

Petersen

2007

Neurocomputing

Self Cite

View full text Add to dashboard Cite

show abstract

“…EP projects the a-posteriori estimation on a Gaussian distribution with moment matching, and thus obtains a similar message update rule as Gaussian message passing (GMP) [14]- [18]. The potential connection between AMP and EP was first shown in [19], [20], in which the fixed points of EP and AMP were shown to be consistent. An EP-based AMP was proposed in [21].…”

Section: Introductionmentioning

confidence: 99%

A New Insight Into GAMP and AMP

Liu

Huang

et al. 2019

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

A concise expectation propagation (EP) based message passing algorithm (MPA) is derived for the general measurement channel. By neglecting some high-order infinitesimal terms, the EP-MPA is proven to be equivalent to the Generalized Approximate Message Passing (GAMP), which exploits central limit theorem and Taylor expansion to simplify the belief propagation process. Furthermore, for additive white gaussian noise measurement channels, EP-MPA is proven to be equivalent to the AMP. Such intrinsic equivalence between EP and GAMP/AMP offers a new insight into GAMP and AMP via a unified message passing rule for non-linear processing, and may provide clues towards building new MPAs in solving more general non-linear problems.

show abstract

“…This approach turns out to be equivalent to that in [6] when applied to turbo-equalization [5]. In [7] a combined use of Gaussian expectation propagation (EP) [8], [9] and BP is proposed. The use of EP, however, leads to an unstable algorithm due to the fact that computed Gaussian EP messages may have a negative variance.…”

Section: Introductionmentioning

confidence: 99%

Iterative Receiver Design for ISI Channels Using Combined Belief- and Expectation-Propagation

Sun

Zhang

Wang

et al. 2015

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

Abstract-In this letter, a message-passing algorithm that combines belief propagation and expectation propagation is applied to design an iterative receiver for intersymbol interference channels. We detail the derivation of the messages passed along the nodes of a vector-form factor graph representing the underlying probabilistic model. We also present a simple but efficient method to cope with the "negative variance" problem of expectation propagation. Simulation results show that the proposed algorithm outperforms, in terms of bit-error-rate and convergence rate, a LMMSE turbo-equalizer based on Gaussian message passing with the same order of computational complexity.

show abstract

Approximate inference techniques with expectation constraints

Cited by 42 publications

References 19 publications

Flexible and efficient implementations of Bayesian independent component analysis

Flexible and efficient implementations of Bayesian independent component analysis

A New Insight Into GAMP and AMP

Iterative Receiver Design for ISI Channels Using Combined Belief- and Expectation-Propagation

Contact Info

Product

Resources

About