Thermodynamically consistent physics-informed neural networks for hyperbolic systems

Patel, Ravi G.; Manickam, Indu; Trask, Nathaniel; Wood, Mitchell; Lee, Myoungkyu; Tomaš, Ignacio; Cyr, Eric C

doi:10.48550/arxiv.2012.05343

Cited by 4 publications

(6 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Physics-Guided Machine Learning. Physics-guided machine learning is becoming an increasingly common method for solving problems in a wide variety of physics dependent fields such as fluid mechanics [7,20,28,34], electromagnetism [19,26,37], thermodynamic modeling [2,16,27,36], and even in medical engineering [29]. By imposing physical constraints to respect any symmetries [35], invariances [20], or conservation principles [4], researchers are able to constrain the space of admissible solutions to a manageable size even with a few hundred data-points.…”

Section: Related Workmentioning

confidence: 99%

Physics-Guided Problem Decomposition for Scaling Deep Learning of High-dimensional Eigen-Solvers: The Case of Schrödinger's Equation

Srivastava¹,

Olin²,

Podolskiy³

et al. 2022

Preprint

View full text Add to dashboard Cite

Given their ability to effectively learn non-linear mappings and perform fast inference, deep neural networks (NNs) have been proposed as a viable alternative to traditional simulation-driven approaches for solving high-dimensional eigenvalue equations (HDEs), which are the foundation for many scientific applications. Unfortunately, for the learned models in these scientific applications to achieve generalization, a large, diverse, and preferably annotated dataset is typically needed and is computationally expensive to obtain. Furthermore, the learned models tend to be memory-and compute-intensive primarily due to the size of the output layer. While generalization, especially extrapolation, with scarce data has been attempted by imposing physical constraints in the form of physics loss, the problem of model scalability has remained.In this paper, we alleviate the compute bottleneck in the output layer by using physics knowledge to decompose the complex regression task of predicting the high-dimensional eigenvectors into multiple simpler sub-tasks, each of which are learned by a simple "expert" network. We call the resulting architecture of specialized experts Physics-Guided Mixture-of-Experts (PG-MoE). We demonstrate the efficacy of such physics-guided problem decomposition for the case of the Schrödinger Equation in Quantum Mechanics. Our proposed PG-MoE model predicts the ground-state solution, i.e., the eigenvector that corresponds to the smallest possible eigenvalue. The model is 150× smaller than the network trained to learn the complex task while being competitive in generalization. To improve the generalization of the PG-MoE, we also employ a physics-guided loss function based on variational energy, which by quantum mechanics principles is minimized iff the output is the ground-state solution. CCS CONCEPTS• Computing methodologies → Learning paradigms; Neural networks; Supervised learning by regression.

show abstract

Section: Related Workmentioning

confidence: 99%

Physics-Guided Problem Decomposition for Scaling Deep Learning of High-dimensional Eigen-Solvers: The Case of Schrödinger's Equation

Srivastava¹,

Olin²,

Podolskiy³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To perform Gaussian process regression, we suppose that the behavior of the function of interest is described by some covariance function k(x, x ). The posterior mean prediction (7) in GPR for a function u at a point x, given data (X, y) = {(x i , y i )} N i=1 , can be written as…”

Section: Eigenfunction Expansion Kernel Functions For Boundary Condit...mentioning

confidence: 99%

“…the mean and variance of the GP posterior prediction for f * = f (x * ), formulas (7) and (8), respectively, can be written as…”

Section: Combining Boundary Value and Linear Pde Constraintsmentioning

confidence: 99%

“…Physics-informed machine learning models which embed physical constraints are a highly active area of research [1,2,3,4]. Constraints for deep neural networks typically take the form of penalty terms in the loss functions to steer the model towards a more physically consistent one during training [5,6,7]. While simple to implement, it is then difficult to quantify the violation of the constraints when extrapolating.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Gaussian Process Regression constrained by Boundary Value Problems

Gulian¹,

Frankel²,

Swiler³

2020

Preprint

View full text Add to dashboard Cite

We develop a framework for Gaussian processes regression constrained by boundary value problems. The framework may be applied to infer the solution of a well-posed boundary value problem with a known second-order differential operator and boundary conditions, but for which only scattered observations of the source term are available. Scattered observations of the solution may also be used in the regression. The framework combines co-kriging with the linear transformation of a Gaussian process together with the use of kernels given by spectral expansions in eigenfunctions of the boundary value problem. Thus, it benefits from a reduced-rank property of covariance matrices. We demonstrate that the resulting framework yields more accurate and stable solution inference as compared to physics-informed Gaussian process regression without boundary condition constraints.

show abstract

“…A number of scientific machine learning (ML) tasks seek to discover a dynamical system whose solution is consistent with data (e.g. constitutive modeling (Patel et al 2020;Karapiperis et al 2021;Ghnatios et al 2019;Masi et al 2021), reduced-order modeling ; Lee and Carlberg 2021;Wan et al 2018), physics-informed machine learning (Karniadakis et al 2021;Wu, Xiao, and Paterson 2018), and surrogates for performing optimal control (Alexopoulos, Nikolakis, and Chryssolouris 2020)). A major challenge for this class of problems is the preservation of both numerical stability and physical realizability when performing out of distribution inference (i.e.…”

Section: Introductionmentioning

confidence: 99%

Structure-preserving Sparse Identification of Nonlinear Dynamics for Data-driven Modeling

Lee,

Trask,

Stinis

2021

Preprint

View full text Add to dashboard Cite

Discovery of dynamical systems from data forms the foundation for data-driven modeling and recently, structurepreserving geometric perspectives have been shown to provide improved forecasting, stability, and physical realizability guarantees. We present here a unification of the Sparse Identification of Nonlinear Dynamics (SINDy) formalism with neural ordinary differential equations. The resulting framework allows learning of both "black-box" dynamics and learning of structure preserving bracket formalisms for both reversible and irreversible dynamics. We present a suite of benchmarks demonstrating effectiveness and structure preservation, including for chaotic systems.

show abstract

Thermodynamically consistent physics-informed neural networks for hyperbolic systems

Cited by 4 publications

References 63 publications

Physics-Guided Problem Decomposition for Scaling Deep Learning of High-dimensional Eigen-Solvers: The Case of Schrödinger's Equation

Physics-Guided Problem Decomposition for Scaling Deep Learning of High-dimensional Eigen-Solvers: The Case of Schrödinger's Equation

Gaussian Process Regression constrained by Boundary Value Problems

Structure-preserving Sparse Identification of Nonlinear Dynamics for Data-driven Modeling

Contact Info

Product

Resources

About