Ensemble Kalman inversion for sparse learning of dynamical systems from time-averaged data

Training Physics‐Based Machine‐Learning Parameterizations With Gradient‐Free Ensemble Kalman Methods

“…On the other hand, UKI provides information about parametric uncertainty and correlations, which can be used to improve models at the process level, and to rapidly compare the added value of increasingly precise observing systems. Other ensemble Kalman methods, such as the sparsity‐inducing EKI (Schneider et al., 2020) or the ensemble Kalman sampler (Garbuno‐Inigo et al., 2020), can provide solutions to the inverse problem with other useful properties. In addition, all these ensemble methods generate parameter‐output pairs that can be used to train emulators for uncertainty quantification that can capture non‐Gaussian posteriors (Cleary et al., 2021).…”

“…The inset in Figure 8b shows how the higher‐complexity hybrid model moderately overfits to the training set after ∼10 epochs, a behavior that is not observed with the empirical model. Hence, in the low‐data regime ( d ≲ p ), adoption of techniques such as early stopping (Prechelt, 1998) or sparsity‐inducing regularization (Schneider et al., 2020) becomes necessary. The compact support property of EKI, which mandates that the solution be in the linear span of the initial ensemble, also regularizes the learned hybrid model with decreasing J ; for J = 50 < p overfitting is significantly reduced.…”

An Efficient Bayesian Approach to Learning Droplet Collision Kernels: Proof of Concept Using “Cloudy,” a New n‐Moment Bulk Microphysics Scheme

“…Methods for quantifying structural uncertainty are less well developed than those for parametric uncertainty, but an established approach is to model the structural error as a Gaussian process at the interface of model and data (Kennedy & O’Hagan, 2000). An alternative is to use Gaussian processes or other machine learning techniques—for example, neural networks or learning from a dictionary of candidate terms (Brunton et al., 2016; Schneider et al., 2021)—directly where structural model errors actually occur, for example, in the collision kernel. In our example, the direct correspondence of the collision and breakup kernels between Cloudy and PySDM allowed us to instead use a simple additive bias term.…”

Ensemble‐Based Experimental Design for Targeting Data Acquisition to Inform Climate Models

“…One interpretation of this added noise is that it plays the role of an artificial corruption of $${\mathcal{S}}_{T}\left({\boldsymbol{\theta }}^{{\dagger}};k\right)$$, with unbiased model error $${\delta }_{k}$$ that plays the same role as additional observational noise (Kennedy & O’Hagan, 2001). One can obtain unbiased $${\delta }_{k}$$ by inclusion of models for structural model error within $${\mathcal{S}}_{T}$$, for example, learned error models that enforce conservation laws and sparsity (M. E. Levine & Stuart, 2021; Schneider et al., 2022). The inverse problem can be written as $$${\boldsymbol{z}}_{k}={\mathcal{S}}_{\infty }(\boldsymbol{\theta };k)+{\gamma }_{k},\qquad {\gamma }_{k}\sim N\left(0,{W}_{k}({\Sigma}(\boldsymbol{\theta })+{\Delta}){W}_{k}^{T}\right).$$$ …”

“…One interpretation of this added noise is that it plays the role of an artificial corruption of  † ; , with unbiased model error 𝐴𝐴 𝐴𝐴𝑘𝑘 that plays the same role as additional observational noise (Kennedy & O'Hagan, 2001). One can obtain unbiased 𝐴𝐴 𝐴𝐴𝑘𝑘 by inclusion of models for structural model error within 𝐴𝐴 𝑇𝑇 , for example, learned error models that enforce conservation laws and sparsity (M. E. Levine & Stuart, 2021;Schneider et al, 2022). The inverse problem 2 can be written as…”