Expectation Propagation in the Large Data Limit

Dehaene, Guillaume; Barthelmé, Simon

doi:10.1111/rssb.12241

Cited by 21 publications

(21 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In an effort to better understand the asymptotic behavior of the context-aware filter for systems with no dynamics, in this section, we analyze the effect of a single update in the limit. In particular, we show that as more data are available, discrete updates converge to a Newton-method-like step (this result is similar to a recent result about the limit behavior of EP [47]).…”

Section: Convergence Of "Site" Approximationssupporting

confidence: 87%

“…However, as shown in Section VI, simulations suggest that the estimates do converge to the true state. Furthermore, similar convergence results exist for the EP algorithm (which also contains a Gaussian approximation), namely, EP converges to the true state for strongly log-concave observation models [46] (the probit model is log-concave but is not strongly log-concave); and in the limit, EP has a fixed point at the true state if the observation model has bounded derivatives [47] (true for the probit model). Thus, it is likely that the context-aware filter's mean also converges to the true state but we leave proving this result for future work.…”

Section: B Covariance Matrix Convergence For Nonmoving Systemmentioning

confidence: 77%

See 1 more Smart Citation

Continuous Estimation Using Context-Dependent Discrete Measurements

Ivanov

Atanasov

Pajić

et al. 2019

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

This paper considers the problem of continuous state estimation from discrete context-based measurements. Context measurements provide binary information as obtained from the system's environment, e.g., a medical alarm indicating that a vital sign is above a certain threshold. Since they provide state information, these measurements can be used for estimation purposes, similar to standard continuous measurements, especially when standard sensors are biased or attacked. Context measurements are assumed to have a known probability of occurring given the state; in particular, we focus on the probit function to model threshold-based measurements, such as the medical-alarm scenario. We develop a recursive context-aware filter by approximating the posterior distribution with a Gaussian distribution with the same first two moments as the true posterior. We show that the filter's expected uncertainty is bounded when the probability of receiving context measurements is lower bounded by some positive number for all system states. Furthermore, we provide an observability-like result-all eigenvalues of the filter's covariance matrix converge to 0 after repeated updates if and only if a persistence of excitation condition holds for the context measurements. Finally, in addition to simulation evaluations, we applied the filter to the problem of estimating a patient's blood oxygen content during surgery using real-patient data.

show abstract

Section: Convergence Of "Site" Approximationssupporting

confidence: 87%

Section: B Covariance Matrix Convergence For Nonmoving Systemmentioning

confidence: 77%

Continuous Estimation Using Context-Dependent Discrete Measurements

Ivanov

Atanasov

Pajić

et al. 2019

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

show abstract

“…A review of all these classes of algorithms can be found in Korobilis and Pettenuzzo (2020). A few representative works relying on such algorithms are Dehaene and Barthelmé (2018), Kim and Wand (2016), Korobilis (2021), Liu et al (2019), Wainwright and Jordan (2008) and Zou et al (2016).…”

Section: Other Approximate Algorithmsmentioning

confidence: 99%

Bayesian Approaches to Shrinkage and Sparse Estimation

Korobilis¹,

Shimizu²

2021

Preprint

View full text Add to dashboard Cite

In all areas of human knowledge, datasets are increasing in both size and complexity, creating the need for richer statistical models. This trend is also true for economic data, where high-dimensional and nonlinear/nonparametric inference is the norm in several fields of applied econometric work. The purpose of this paper is to introduce the reader to the world of Bayesian model determination, by surveying modern shrinkage and variable selection algorithms and methodologies. Bayesian inference is a natural probabilistic framework for quantifying uncertainty and learning about model parameters, and this feature is particularly important for inference in modern models of high dimensions and increased complexity.We begin with a linear regression setting in order to introduce various classes of priors that lead to shrinkage/sparse estimators of comparable value to popular penalized likelihood estimators (e.g. ridge, lasso). We explore various methods of exact and approximate inference, and discuss their pros and cons. Finally, we explore how priors developed for the simple regression setting can be extended in a straightforward way to various classes of interesting econometric models. In particular, the following case-studies are considered, that demonstrate application of Bayesian shrinkage and variable selection strategies to popular econometric contexts: i) vector autoregressive models; ii) factor models; iii) time-varying parameter regressions; iv) confounder selection in treatment effects models; and v) quantile regression models. A MATLAB package and an accompanying technical manual allow the reader to replicate many of the algorithms described in this review.

show abstract

“…Since its first appearance in 2001, EP has found many successful applications in practice, and it is reported to be very accurate, e.g., for Gaussian processes [36], and electrical impedance tomography with sparsity prior [19]. However, the theoretical understanding of EP remains quite limited [14,13].…”

Section: Approximate Inference By Expectation Propagationmentioning

confidence: 99%

Expectation propagation for Poisson data

2019

View full text Add to dashboard Cite

The Poisson distribution arises naturally when dealing with data involving counts, and it has found many applications in inverse problems and imaging. In this work, we develop an approximate Bayesian inference technique based on expectation propagation for approximating the posterior distribution formed from the Poisson likelihood function and a Laplace type prior distribution, e.g., the anisotropic total variation prior. The approach iteratively yields a Gaussian approximation, and at each iteration, it updates the Gaussian approximation to one factor of the posterior distribution by moment matching. We derive explicit update formulas in terms of one-dimensional integrals, and also discuss stable and efficient quadrature rules for evaluating these integrals. The method is showcased on two-dimensional PET images.

show abstract

Expectation Propagation in the Large Data Limit

Cited by 21 publications

References 18 publications

Continuous Estimation Using Context-Dependent Discrete Measurements

Continuous Estimation Using Context-Dependent Discrete Measurements

Bayesian Approaches to Shrinkage and Sparse Estimation

Expectation propagation for Poisson data

Contact Info

Product

Resources

About