Robust, Automated, and Accurate Black-box Variational Inference

Welandawe, Manushi; Andersen, Michael Riis; Vehtari, Aki; Huggins, Jonathan H.

doi:10.48550/arxiv.2203.15945

Cited by 2 publications

(2 citation statements)

References 18 publications

(45 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, the cumbersome derivation process of the CAVI method used in VBAKF is not friendly to high-dimensional dynamic systems [41]. To overcome the limitations of VBAKF and simplify the algorithm derivation, the BBVIAKF is proposed, which approximates gradients through sampling methods [42][43][44][45]. In particular, an SVGD based Bayesian inference approach put forward recently has shown more promising prospects in estimating target distribution [46,47].…”

Section: Introductionmentioning

confidence: 99%

A generalized data assimilation architecture of digital twin for complex process industrial systems

Zhao,

Jiang,

Cai

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

As one of the critical cores of digital twin (DT), data assimilation (DA) can maintain consistency and synchronization between DT and physical system. Kalman filtering is a common DA method, but its estimation performance is deteriorated by factors such as model inaccuracy and time-varying noise covariance in practical applications. The errors caused by these multiple uncertainties are all coupled in the measurements, which augments the difficulty for DT to obtain physical system information. In order to tackle the DA problem with multiple uncertainties, this paper proposes a generalized DA architecture of DT for sophisticated process industry. First, combining Stein variational gradient descent (SVGD) and nonlinear Bayesian filtering paradigm, a recursive estimation framework is established, which has higher accuracy in estimating the noise covariance compared to traditional methods. Second, to effectively deal with model inaccuracy by using the filtering residual containing time-varying noise, we propose a neural network and modified wavelet-based model error compensation (NNMW-MEC) block. Based on modified wavelet technique, the filtering residual denoising (FRD) built in NNMW-MEC can better cope with time-varying noise compared to existing wavelets, and extract the low-frequency signal involving model error information from noisy residual smoothly. In addition, because of the neural network-based state-compensation (NNSC) subblock, NNMW-MEC has more outstanding ability in compensating the state deviations with large changing range. Finally, we take the boiler system in a coal-fired power plant as an example to verify the effectiveness of our architecture. Experimental results show that the DA architecture proposed in this paper can improve the estimation performance of DT under inaccurate model and uncertain noise statistics.

show abstract

Section: Introductionmentioning

confidence: 99%

A generalized data assimilation architecture of digital twin for complex process industrial systems

Zhao,

Jiang,

Cai

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…One such algorithm is variational inference, which has recently emerged as a popular alternative to the classical Markov chain Monte Carlo (MCMC) algorithms (Blei et al, 2017;Jordan et al, 1999). Unlike MCMC that relies on sampling, variational inference infers the posterior by solving a constrained optimization problem, and scales to large datasets by leveraging modern advances in stochastic optimization (Ho man et al, 2013;Welandawe et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection

Bhatia¹,

Kuang²,

Ma³

et al. 2022

Preprint

View full text Add to dashboard Cite

Variational inference has recently emerged as a popular alternative to the classical Markov chain Monte Carlo (MCMC) in large-scale Bayesian inference. The core idea of variational inference is to trade statistical accuracy for computational e ciency. It aims to approximate the posterior, reducing computation costs but potentially compromising its statistical accuracy. In this work, we study this statistical and computational trade-o in variational inference via a case study in inferential model selection. Focusing on Gaussian inferential models (also known as variational approximating families) with diagonal plus low-rank precision matrices, we initiate a theoretical study of the trade-o s in two aspects, Bayesian posterior inference error and frequentist uncertainty quanti cation error. From the Bayesian posterior inference perspective, we characterize the error of the variational posterior relative to the exact posterior. We prove that, given a xed computation budget, a lower-rank inferential model produces variational posteriors with a higher statistical approximation error, but a lower computational error; it reduces variances in stochastic optimization and, in turn, accelerates convergence. From the frequentist uncertainty quanti cation perspective, we consider the precision matrix of the variational posterior as an uncertainty estimate. We nd that, relative to the true asymptotic precision, the variational approximation su ers from an additional statistical error originating from the sampling uncertainty of the data. Moreover, this statistical error becomes the dominant factor as the computation budget increases. As a consequence, for small datasets, the inferential model need not be full-rank to achieve optimal estimation error (even with unlimited computation budget). We nally demonstrate these statistical and computational trade-o s inference across empirical studies, corroborating the theoretical ndings.

show abstract

Robust, Automated, and Accurate Black-box Variational Inference

Cited by 2 publications

References 18 publications

A generalized data assimilation architecture of digital twin for complex process industrial systems

A generalized data assimilation architecture of digital twin for complex process industrial systems

Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection

Contact Info

Product

Resources

About