Iterated Kalman methodology for inverse problems

Huang, Daniel Z.; Schneider, Tapio; Stuart, Andrew M.

doi:10.1016/j.jcp.2022.111262

Cited by 27 publications

(22 citation statements)

References 86 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We consider two highly parallelizable gradient-free algorithms to solve the inverse problem defined in Section 2: ensemble Kalman inversion (EKI, Iglesias et al, 2013) and unscented Kalman inversion (UKI, Huang, Schneider, & Stuart, 2022). Both algorithms are based on the extended Kalman filter and draw heavily on Gaussian conditioning for their derivation: underlying their update rules is the approximation of the parameter distribution as Gaussian.…”

Section: Ensemble Kalman Methodsmentioning

confidence: 99%

“…EKI and UKI have been used successfully in a wide variety of inverse problems (Huang, Schneider, & Stuart, 2022; Iglesias, 2016; Iglesias et al., 2013; Kovachki & Stuart, 2019; Xiao et al., 2016). We demonstrate them here in the context of training models that may experience numerical instabilities for a priori unknown parameter combinations, starting with a brief review of the algorithms.…”

Section: Ensemble Kalman Methodsmentioning

confidence: 99%

“…However, for situations where using an ensemble of 2 p + 1 members is tractable, UKI improves upon EKI by providing uncertainty quantification, instead of collapsing to a point estimate. In particular, when updates and are applied to the augmented data‐model relation , UKI ensures that Σ n in the limit n → ∞ converges toward a Gaussian estimate of parametric uncertainty (Huang, Schneider, & Stuart, 2022),

{{\Sigma}}_{\infty }\approx {\mathrm{C}\mathrm{o}\mathrm{v}}_{q}\left({\theta }_{\infty },{\mathcal{G}}_{a,\infty }\right){\left[{\Delta}t\cdot {\mathrm{C}\mathrm{o}\mathrm{v}}_{q}\left({\mathcal{G}}_{a,\infty },{\mathcal{G}}_{a,\infty }\right)+{{\Gamma}}_{a}\right]}^{-1}{\mathrm{C}\mathrm{o}\mathrm{v}}_{q}{\left({\theta }_{\infty },{\mathcal{G}}_{a,\infty }\right)}^{T},

which involves the augmented forward model

{\mathcal{G}}_{a}(\cdot )

and covariance Γ a defined in Section 3.2. Σ ∞ approximates the covariance of the posterior in Equation around m ∞ if the full loss is evaluated at every UKI iteration and Δ t = 1 (Huang, Huang, et al., 2022).…”

Section: Ensemble Kalman Methodsmentioning

confidence: 99%

“…Solutions to the inverse problem are sought using the ensemble and unscented Kalman inversion algorithms (Iglesias et al, 2013;Huang, Schneider, & Stuart, 2022). Emphasis is given to practical aspects of this specific inverse problem, which have not previously been explored in the literature.…”

Section: Introductionmentioning

confidence: 99%

“…However, for situations where using an ensemble of 2p + 1 members is tractable, UKI improves upon EKI by providing uncertainty quantification, instead of collapsing to a point estimate. In particular, when updates ( 27) and ( 28) are applied to the augmented datamodel relation ( 25), UKI ensures that Σ n in the limit n → ∞ converges towards a Gaussian estimate of parametric uncertainty (Huang, Schneider, & Stuart, 2022),…”

mentioning

confidence: 99%

See 4 more Smart Citations

Training Physics‐Based Machine‐Learning Parameterizations With Gradient‐Free Ensemble Kalman Methods

Lopez‐Gomez

Christopoulos

Ervik

et al. 2022

J Adv Model Earth Syst

Self Cite

View full text Add to dashboard Cite

Most machine learning applications in Earth system modeling currently rely on gradientbased supervised learning. This imposes stringent constraints on the nature of the data used for training (typically, residual time tendencies are needed), and it complicates learning about the interactions between machine-learned parameterizations and other components of an Earth system model. Approaching learning about process-based parameterizations as an inverse problem resolves many of these issues, since it allows parameterizations to be trained with partial observations or statistics that directly relate to quantities of interest in long-term climate projections. Here we demonstrate the effectiveness of Kalman inversion methods in treating learning about parameterizations as an inverse problem. We consider two different algorithms: unscented and ensemble Kalman inversion. Both methods involve highly parallelizable forward model evaluations, converge exponentially fast, and do not require gradient computations. In addition, unscented Kalman inversion provides a measure of parameter uncertainty. We illustrate how training parameterizations can be posed as a regularized inverse problem and solved by ensemble Kalman methods through the calibration of an eddy-diffusivity mass-flux scheme for subgrid-scale turbulence and convection, using data generated by large-eddy simulations. We find the algorithms amenable to batching strategies, robust to noise and model failures, and efficient in the calibration of hybrid parameterizations that can include empirical closures and neural networks.

show abstract

Section: Ensemble Kalman Methodsmentioning

confidence: 99%

Section: Ensemble Kalman Methodsmentioning

confidence: 99%

{{\Sigma}}_{\infty }\approx {\mathrm{C}\mathrm{o}\mathrm{v}}_{q}\left({\theta }_{\infty },{\mathcal{G}}_{a,\infty }\right){\left[{\Delta}t\cdot {\mathrm{C}\mathrm{o}\mathrm{v}}_{q}\left({\mathcal{G}}_{a,\infty },{\mathcal{G}}_{a,\infty }\right)+{{\Gamma}}_{a}\right]}^{-1}{\mathrm{C}\mathrm{o}\mathrm{v}}_{q}{\left({\theta }_{\infty },{\mathcal{G}}_{a,\infty }\right)}^{T},

which involves the augmented forward model

{\mathcal{G}}_{a}(\cdot )

Section: Ensemble Kalman Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…However, for situations where using an ensemble of 2p + 1 members is tractable, UKI improves upon EKI by providing uncertainty quantification, instead of collapsing to a point estimate. In particular, when updates ( 27) and ( 28) are applied to the augmented datamodel relation ( 25), UKI ensures that Σ n in the limit n → ∞ converges towards a Gaussian estimate of parametric uncertainty (Huang, Schneider, & Stuart, 2022),…”

mentioning

confidence: 99%

See 3 more Smart Citations