Leveraging Bayesian analysis to improve accuracy of approximate models

Abstract

We focus on improving the accuracy of an approximate model of a multiscale dynamical system that uses a set of parameter-dependent terms to account for the effects of unresolved or neglected dynamics on resolved scales. We start by considering various methods of calibrating and analyzing such a model given a few well-resolved simulations. After presenting results for various point estimates and discussing some of their shortcomings, we demonstrate (a) the potential of hierarchical Bayesian analysis to uncover previously unanticipated physical dependencies in the approximate model, and (b) how such insights can then be used to improve the model. In effect parametric dependencies found from the Bayesian analysis are used to improve structural aspects of the model. While we choose to illustrate the procedure in the context of a closure model for buoyancy-driven, variable-density turbulence, the statistical nature of the approach makes it more generally applicable. Towards addressing issues of increased computational cost associated with the procedure, we demonstrate the use of a neural network based surrogate in accelerating the posterior sampling process and point to recent developments in variational inference as an alternative methodology for greatly mitigating such costs. We conclude by suggesting that modern validation and uncertainty quantification techniques such as the ones we consider have a valuable role to play in the development and improvement of approximate models.

Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations

“…Apart from W in and b in there are numerous hyperparameters which require tuning, such as the regularization parameter β, the inflation parameter α, the size and the variance of the initial ensemble, as well as the assumed observational error covariance Γ. To obtain optimal performance of RAFD these would need to be tuned which can be done by additional optimization procedures (Nadiga et al, 2019).…”

Uncertainty Quantification of Ocean Parameterizations: Application to the K‐Profile‐Parameterization for Penetrative Convection

“…These estimates are only as good as the choice of likelihood function, much like optimal parameters are only as good as the choice of the loss function. See, for example, Morrison et al (2020), Nadiga et al (2019), Schneider et al (2017), Sraj et al (2016), Urrego‐Blanco et al (2016), Zedler et al (2012), and van Lier‐Walqui et al (2012) for definitions of likelihoods in various geophysical/fluid dynamical contexts. In Appendix we discuss in detail the rationale for the choices made in this paper.…”

Hydrodynamic and Radiographic Toolbox (HART)

“…While it is clear that a Bayesian framework is well suited to comprehensively quantify uncertainty in this context, the chief impediment to a practical realization of such an analysis is the immense computational cost of the physics-based forward model C f wd . In the first approach, we will include the deep learning based forward model in a Bayesian inference framework [83] to establish uncertainties in the reconstruction. Again, recent improvements in computational inference methods will be brought to bear on this issue e.g., by using Hamiltonian Monte Carlo whose computational cost only scales as 10 3 × C f wd as compared to Metropolis based schemes whose cost scales as 10 6 ×C f wd [7].…”