Optimization Based Methods for Partially Observed Chaotic Systems

Abstract

In this paper we consider filtering and smoothing of partially observed chaotic dynamical systems that are discretely observed, with an additive Gaussian noise in the observation. These models are found in a wide variety of real applications and include the Lorenz 96' model. In the context of a fixed observation interval T , observation time step h and Gaussian observation variance σ 2 Z , we show under assumptions that the filter and smoother are well approximated by a Gaussian with high probability when h and σ 2 Z h are sufficiently small. Based on this result we show that the maximum a posteriori (MAP) estimators are asymptotically optimal in mean square error as σ 2 Z h tends to 0. Given these results, we provide a batch algorithm for the smoother and filter, based on Newton's method, to obtain the MAP. In particular, we show that if the initial point is close enough to the MAP, then Newton's method converges to it at a fast rate. We also provide a method for computing such an initial point. These results contribute to the theoretical understanding of widely used 4D-Var data assimilation method. Our approach is illustrated numerically on the Lorenz 96' model with state vector up to 1 million dimensions, with code running in the order of minutes. To our knowledge the results in this paper are the first of their type for this class of models.

A 4D-Var Method with Flow-Dependent Background Covariances for the Shallow-Water Equations

“…In [52], it is shown that the MAP estimator for the smoother has some desirable theoretical properties. In particular, in the small-noise/high-frequency scenario (with T fixed, and σ √ h → 0), under some conditions, MAP is asymptotically optimal in mean-square error.…”

“…The second reason is that a Gaussian distribution propagated through non-linear dynamics for longer and longer intervals of length bT becomes highly non-Gaussian for large values of b, so the resulting background distribution can lead to poorer results than using smaller values of b. Reminiscent to 4D-Var, at time t (m−b)k we always start off the procedure with the same background covariance B 0 . In [52] it was shown -under certain assumptions -that for a class of non-linear dynamical systems, for a fixed observation window T , if R i = O(σ 2 ) and σ √ h is sufficiently small (h is the observation time step) then the smoothing and filtering distributions can indeed be well approximated by Gaussian laws. Following the ideas behind (3.5), an approximation of the Hessian of J, evaluated at the MAP given data y • (m−1)k:mk−1 is given as…”

“…In fact, this was confirmed during our simulations, and we have found that increasing T beyond a certain range did not improve the performance, while increasing b has resulted in an improvement in general up to a certain point. Based on [52], we expect the Gaussian approximation underlying the choice (3.10) to hold if the observation noise is sufficiently small, the observation frequency is sufficiently high, and the system dynamics and the observations are not too non-linear.…”

“…In [52], the MAP was also shown to be asymptotically optimal in MSE for a class of nonlinear systems under small obervation noise or high observation frequency scenarios for a fixed observation window length. The reason for this is that in these situations, the posterior distribution near the true initial position can be well approximated by a suitably chosen Gaussian distribution, and hence the MAP estimator is close to the posterior mean, which is optimal in MSE.…”

“…Based on this theoretical result, we expect the 4D-Var method to perform well in the small-noise/high-frequency setting, when the dynamical system and the observations are not too non-linear. In this setting, based on the results of [52], one expects the posterior distribution to be approximately Gaussian. If the conditions are not met, it is likely that the Gaussian approximation of the smoothing distribution is no longer accurate and the MAP is no longer necessarily close to optimal in MSE.…”