Bayesian Nonlinear Filtering via Information Geometric Optimization

Li, Yubo; Cheng, Yongqiang; Li, Xiang; Wang, Hongqiang; Hua, Xiaoqiang; Qin, Yuliang

doi:10.3390/e19120655

Cited by 10 publications

(4 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this light, the present variational Bayesian derivation of the Kalman filter shows as a way to 1) preserve its classical connections to Bayesian inference for the exact (i.e., linear) case [49], 2) understand the Kalman filter as the optimal linear estimator for non-linear systems [17,85] in terms of a fixed-form (variational) Gaussian approximation of arbitrary distributions [73,50] and 3) link the filter to more recent (information) geometric treatments of probabilistic inference [65,72].…”

Section: Discussionmentioning

confidence: 99%

Kalman filters as the steady-state solution of gradient descent on variational free energy

Baltieri¹,

Isomura²

2021

Preprint

View full text Add to dashboard Cite

The Kalman filter is an algorithm for the estimation of hidden variables in dynamical systems under linear Gauss-Markov assumptions with widespread applications across different fields. Recently, its Bayesian interpretation has received a growing amount of attention especially in neuroscience, robotics and machine learning. In neuroscience, in particular, models of perception and control under the banners of predictive coding, optimal feedback control, active inference and more generally the so-called Bayesian brain hypothesis, have all heavily relied on ideas behind the Kalman filter. Active inference, an algorithmic theory based on the free energy principle, specifically builds on approximate Bayesian inference methods proposing a variational account of neural computation and behaviour in terms of gradients of variational free energy. Using this ambitious framework, several works have discussed different possible relations between free energy minimisation and standard Kalman filters. With a few exceptions, however, such relations point at a mere qualitative resemblance or are built on a set of very diverse comparisons based on purported differences between free energy minimisation and Kalman filtering. In this work, we present a straightforward derivation of Kalman filters consistent with active inference via a variational treatment of free energy minimisation in terms of gradient descent. The approach considered here offers a more direct link between models of neural dynamics as gradient descent and standard accounts of perception and decision making based on probabilistic inference, further bridging the gap between hypotheses about neural implementation and computational principles in brain and behavioural sciences.

show abstract

Section: Discussionmentioning

confidence: 99%

Kalman filters as the steady-state solution of gradient descent on variational free energy

Baltieri¹,

Isomura²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…2.1.3]. Moreover, a solution of the minimization problem (4) with C(ϕ) ≡ d KL (f (X; ϕ), f (X)) corresponds to the maximum likelihood estimate of the parameters ϕ [27]. This analogy elucidates the connection between Bayesian inference and information geometry.…”

Section: Preliminariesmentioning

confidence: 93%

“…A closure approximation is needed to render the cross-correlation term w (t)π (X, t) computable. Subtracting (27) from (26), we obtain an equation for random fluctuations π (X, t), ∂π ∂t + µ a ∂π ∂X = ∂( s (X, t)π (X, t) − s π) ∂X , subject to π (X, t = 0) = 0.…”

Section: A3 MD For the Langevin Equation With Colored Noisementioning

confidence: 99%

Information-geometry of physics-informed statistical manifolds and its use in data assimilation

Boso,

Tartakovsky

2021

Preprint

View full text Add to dashboard Cite

The data-aware method of distributions (DA-MD) is a low-dimension data assimilation procedure to forecast the behavior of dynamical systems described by differential equations. It combines sequential Bayesian update with the MD, such that the former utilizes available observations while the latter propagates the (joint) probability distribution of the uncertain system state(s). The core of DA-MD is the minimization of a distance between an observation and a prediction in distributional terms, with prior and posterior distributions constrained on a statistical manifold defined by the MD. We leverage the information-geometric properties of the statistical manifold to reduce predictive uncertainty via data assimilation. Specifically, we exploit the information geometric structures induced by two discrepancy metrics, the Kullback-Leibler divergence and the Wasserstein distance, which explicitly yield natural gradient descent. To further accelerate optimization, we build a deep neural network as a surrogate model for the MD that enables automatic differentiation. The manifold's geometry is quantified without sampling, yielding an accurate approximation of the gradient descent direction. Our numerical experiments demonstrate that accounting for the information-geometry of the manifold significantly reduces the computational cost of data assimilation by facilitating the calculation of gradients and by reducing the number of required iterations. Both storage needs and computational cost depend on the dimensionality of a statistical manifold, which is typically small by MD construction. When convergence is achieved, the Kullback-Leibler and L 2 Wasserstein metrics have similar performances, with the former being more sensitive to poor choices of the prior.

show abstract

“…The main challenges associated with passive multiple underwater target tracking are that the passively obtained information is highly nonlinear [5][6][7], the targets' range may be unobservable, and the data association uncertainty between passive measurements and targets is complicated.…”

Section: Introductionmentioning

confidence: 99%

Passive Tracking of Multiple Underwater Targets in Incomplete Detection and Clutter Environment

Ali

et al. 2021

Entropy

View full text Add to dashboard Cite

A major advantage of the use of passive sonar in the tracking multiple underwater targets is that they can be kept covert, which reduces the risk of being attacked. However, the nonlinearity of the passive Doppler and bearing measurements, the range unobservability problem, and the complexity of data association between measurements and targets make the problem of underwater passive multiple target tracking challenging. To deal with these problems, the cardinalized probability hypothesis density (CPHD) recursion, which is based on Bayesian information theory, is developed to handle the data association uncertainty, and to acquire existing targets’ numbers and states (e.g., position and velocity). The key idea of the CPHD recursion is to simultaneously estimate the targets’ intensity and the probability distribution of the number of targets. The CPHD recursion is the first moment approximation of the Bayesian multiple targets filter, which avoids the data association procedure between the targets and measurements including clutter. The Bayesian-filter-based extended Kalman filter (EKF) is applied to deal with the nonlinear bearing and Doppler measurements. The experimental results show that the EKF-based CPHD recursion works well in the underwater passive multiple target tracking system in cluttered and noisy environments.

show abstract

Bayesian Nonlinear Filtering via Information Geometric Optimization

Cited by 10 publications

References 35 publications

Kalman filters as the steady-state solution of gradient descent on variational free energy

Kalman filters as the steady-state solution of gradient descent on variational free energy

Information-geometry of physics-informed statistical manifolds and its use in data assimilation

Passive Tracking of Multiple Underwater Targets in Incomplete Detection and Clutter Environment

Contact Info

Product

Resources

About