Systematic variational method for statistical nonlinear state and parameter estimation

Abstract

In statistical data assimilation one evaluates the conditional expected values, conditioned on measurements, of interesting quantities on the path of a model through observation and prediction windows. This often requires working with very high dimensional integrals in the discrete time descriptions of the observations and model dynamics, which become functional integrals in the continuous-time limit. Two familiar methods for performing these integrals include (1) Monte Carlo calculations and (2) variational approximations using the method of Laplace plus perturbative corrections to the dominant contributions. We attend here to aspects of the Laplace approximation and develop an annealing method for locating the variational path satisfying the Euler-Lagrange equations that comprises the major contribution to the integrals. This begins with the identification of the minimum action path starting with a situation where the model dynamics is totally unresolved in state space, and the consistent minimum of the variational problem is known. We then proceed to slowly increase the model resolution, seeking to remain in the basin of the minimum action path, until a path that gives the dominant contribution to the integral is identified. After a discussion of some general issues, we give examples of the assimilation process for some simple, instructive models from the geophysical literature. Then we explore a slightly richer model of the same type with two distinct time scales. This is followed by a model characterizing the biophysics of individual neurons.

Precision annealing Monte Carlo methods for statistical data assimilation and machine learning

“…We now turn to the precision annealing method suggested in [29,30,50]. It is used here to facilitate the search for the global minimum of the action A(X) as we gradually increase the model precision parameter R f .…”

“…The Laplace-method evaluations of expected value integrals is discussed in [27][28][29][30]. They do not sample from π(X | Y) away from its maximum.…”

“…Precision Annealing arises from the idea that if we choose an initial path for a Monte Carlo or a variational optimization [27][28][29][30] at the global minimum for small R f as we slowly increase R f , we will stay in a region of phase space where our sampling will arrive at the smallest minimum of the action, for a variational calculation, or sample the neighborhood of the maximum of π(X, P), for a Monte Carlo search. We adopted the following annealing schedule for R f :…”

Quasi static ensemble variational data assimilation

“…On the other hand, with traditional 4D-Var, the prior covariance matrix is taken as static. This is appropriate when, as in Pires et al (1996), Swanson et al (1998) or Ye et al (2015), only one cycle of assimilation is considered. But this limits the dynamical transfer of error statistics from one cycle to the next.…”

“…Pires et al (1996) propose the quasi-static (QS) minimization in a 4D-Var context: as the observations are progressively added to the cost function, the starting point (or first guess) of the 4D-Var minimization is also gradually updated. Ye et al (2015) propose to gradually increase the model error covariances in the weak-constraint 4D-Var cost function in a minimization over an entire trajectory; this way the model nonlinearity is gradually introduced into the cost function (see also Judd et al, 2004). They also propose to parallelize this minimization over 25 multiple starting points to increase the chance to locate the global minimum.…”