Bayesian differential programming for robust systems identification under uncertainty

Yang, Yibo; Bhouri, Mohamed Aziz; Perdikaris, Paris

doi:10.1098/rspa.2020.0290

Cited by 27 publications

(28 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, where p(σ 2 θ ), p(σ 2 u ) are hyperpriors; see, e.g., [31,87]. Note that, following this approach, the likelihood term p(D|θ, σ 2 θ , σ 2 u ) does not depend explicitly on σ 2 θ , and thus σ 2 θ only appears in the prior part of the posterior.…”

Section: B21 Markov Chain Monte Carlo (Mcmc)mentioning

confidence: 99%

Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

Psaros¹,

Meng²,

Zou³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. To demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed, which we will make available as open-source library of all codes included in this framework.

show abstract

Section: B21 Markov Chain Monte Carlo (Mcmc)mentioning

confidence: 99%

Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

Psaros¹,

Meng²,

Zou³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For each experiment, we run the SINDy-SA method to solve the sparse regression problem, consisting basically of the three steps described in Subsection 2.2: the ridge regression for estimating the sparse vectors of coefficients b Ξ τ , the computation of SSE between the derivatives Ẋ and c Ẋτ , and the sensitivity analysis to eliminate less important terms from the governing equations. Alternatively, in our general framework for system identification, the SINDy-SA method can be replaced with other approaches for identifying nonlinear dynamical systems [2,7,6,25,20,35,37,29].…”

Section: General Framework For System Identificationmentioning

confidence: 99%

“…Machine learning methods have been commonly used to understand behaviors, recognize patterns and make predictions from experimental data. Furthermore, another application of these methods, which has become popular in recent years, is the structural identification of mathematical models [2,7,6,25,20,35,37,29]. These models, in turn, help to interpret the dynamics and allow the use of tools for mathematical analysis.…”

Section: Introductionmentioning

confidence: 99%

SINDy-SA: Enhancing Nonlinear System Identification with Sensitivity Analysis

Naozuka

Rocha

Silva

et al. 2022

Preprint

View full text Add to dashboard Cite

Machine learning methods have revolutionized studies in several areas of knowledge, helping to understand and extract information from experimental data. Recently, these data-driven methods have also been used to discover structures of mathematical models. The sparse identification of nonlinear dynamics (SINDy) method has been proposed with the aim of identifying nonlinear dynamical systems, assuming that the equations have only a few important terms that govern the dynamics. By defining a library of possible terms, the SINDy approach solves a sparse regression problem by eliminating terms whose coefficients are smaller than a threshold. However, the choice of this threshold is decisive for the correct identification of the model structure. In this work, we build on the SINDy method by integrating it with a global sensitivity analysis (SA) technique that allows to classify terms according to their importance in relation to the desired quantity of interest. The proposed SINDy-SA approach thus eliminates the need to define the SINDy threshold. We compare our method with the original SINDy approach employing them in a variety of applications whose simulated data have different behaviors. For each application, we formulate different experimental settings and select the best model for both methods using model selection techniques based on information criteria. The results demonstrate that the SINDy-SA framework is a promising methodology to accurately identify interpretable data-driven models.

show abstract

“…Some directly learn the ODE system from high-fidelity simulation data without using the available low-fidelity models [21], which could lead to the requirement of bigger NNs. Others combine nODEs with model discovery using sparse-regression [22] or only learn the values of parameters in existing closure models [23]. Nearly all existing studies primarily only attempt to address the closure for ROMs.…”

Section: Introductionmentioning

confidence: 99%

Neural closure models for dynamical systems

Gupta

Lermusiaux

2021

Proc. R. Soc. A.

View full text Add to dashboard Cite

Complex dynamical systems are used for predictions in many domains. Because of computational costs, models are truncated, coarsened or aggregated. As the neglected and unresolved terms become important, the utility of model predictions diminishes. We develop a novel, versatile and rigorous methodology to learn non-Markovian closure parametrizations for known-physics/low-fidelity models using data from high-fidelity simulations. The new neural closure models augment low-fidelity models with neural delay differential equations (nDDEs), motivated by the Mori–Zwanzig formulation and the inherent delays in complex dynamical systems. We demonstrate that neural closures efficiently account for truncated modes in reduced-order-models, capture the effects of subgrid-scale processes in coarse models and augment the simplification of complex biological and physical–biogeochemical models. We find that using non-Markovian over Markovian closures improves long-term prediction accuracy and requires smaller networks. We derive adjoint equations and network architectures needed to efficiently implement the new discrete and distributed nDDEs, for any time-integration schemes and allowing non-uniformly spaced temporal training data. The performance of discrete over distributed delays in closure models is explained using information theory, and we find an optimal amount of past information for a specified architecture. Finally, we analyse computational complexity and explain the limited additional cost due to neural closure models.

show abstract

Bayesian differential programming for robust systems identification under uncertainty

Cited by 27 publications

References 59 publications

Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons

SINDy-SA: Enhancing Nonlinear System Identification with Sensitivity Analysis

Neural closure models for dynamical systems

Contact Info

Product

Resources

About