Machine learning of linear differential equations using Gaussian processes

Raissi, Maziar; Perdikaris, Paris; Karniadakis, George Em

doi:10.1016/j.jcp.2017.07.050

Cited by 506 publications

(363 citation statements)

References 31 publications

Supporting

Mentioning

333

Contrasting

Order By: Relevance

“…In Raissi et al [2017b], a data-driven algorithm for learning the coefficients of general parametric linear differential equations from noisy data was introduced, solving a so-called inverse problem. The approach employs GP priors that are tailored to the corresponding and known type of differential operators, resulting in design and algorithmic transparency.…”

Section: Prediction Of Scientific Parameters and Propertiesmentioning

confidence: 99%

Explainable Machine Learning for Scientific Insights and Discoveries

et al. 2020

View full text Add to dashboard Cite

Machine learning methods have been remarkably successful for a wide range of application areas in the extraction of essential information from data. An exciting and relatively recent development is the uptake of machine learning in the natural sciences, where the major goal is to obtain novel scientific insights and discoveries from observational or simulated data. A prerequisite for obtaining a scientific outcome is domain knowledge, which is needed to gain explainability, but also to enhance scientific consistency. In this article we review explainable machine learning in view of applications in the natural sciences and discuss three core elements which we identified as relevant in this context: transparency, interpretability, and explainability. With respect to these core elements, we provide a survey of recent scientific works that incorporate machine learning and the way that explainable machine learning is used in combination with domain knowledge from the application areas. * R. Roscher and J. Garcke contributed equally to this work arXiv:1905.08883v3 [cs.LG]

show abstract

Section: Prediction Of Scientific Parameters and Propertiesmentioning

confidence: 99%

Explainable Machine Learning for Scientific Insights and Discoveries

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Two immediate opportunities for machine learning in multiscale modeling include learning the underlying physics [104] and learning the parameters for a known physics-based problem. Recent examples of learning the underlying physics are the data-driven solution of problems in elasticity [23] and the data-driven discovery of partial differential equations for nonlinear dynamical systems [14,95,98]. This class of problems holds great promise, especially in combination with deep learning, but involves a thorough understanding and direct interaction with the underlying learning machines [100].…”

Section: Motivationmentioning

confidence: 99%

Multiscale Modeling Meets Machine Learning: What Can We Learn?

Peng

Alber

Tepole³

et al. 2020

Arch Computat Methods Eng

Self Cite

222

103

View full text Add to dashboard Cite

Machine learning is increasingly recognized as a promising technology in the biological, biomedical, and behavioral sciences. There can be no argument that this technique is incredibly successful in image recognition with immediate applications in diagnostics including electrophysiology, radiology, or pathology, where we have access to massive amounts of annotated data. However, machine learning often performs poorly in prognosis, especially when dealing with sparse data. This is a field where classical physics-based simulation seems to remain irreplaceable. In this review, we identify areas in the biomedical sciences where machine learning and multiscale modeling can mutually benefit from one another: Machine learning can integrate physics-based knowledge in the form of governing equations, boundary conditions, or constraints to manage ill-posted problems and robustly handle sparse and noisy data; multiscale modeling can integrate machine learning to create surrogate models, identify system dynamics and parameters, analyze sensitivities, and quantify uncertainty to bridge the scales and understand the emergence of function. With a view towards applications in the life sciences, we discuss the state of the art of combining machine learning and multiscale modeling, identify applications and opportunities, raise open questions, and address potential challenges and limitations. This review serves as introduction to a special issue on Uncertainty Quantification, Machine Learning, and Data-Driven Modeling of Biological Systems that will help identify current roadblocks and areas where computational mechanics, as a discipline, can play a significant role. We anticipate that it will stimulate discussion within the community of computational mechanics and reach out to other disciplines including mathematics, statistics, computer science, artificial intelligence, biomedicine, systems biology, and precision medicine to join forces towards creating robust and efficient models for biological systems.

show abstract

“…Similarly, concatenate s 0 and all the s e into a single vector s, defining K as the space of all s that have sub-vector s 0 ∈ Q 3n+1 and all sub-vectors s e ∈ Q 4 . With this notation, Equation 3 can be written more formally as a second order cone program (SOCP):…”

Section: Inextensibility Priormentioning

confidence: 99%

“…Since a number of authors have begun to consider the use of machine/deep learning for problems in traditional computational physics, see e.g. [1,2,3,4,5,6,7,8,9,10,11,12], we are motivated to consider methodologies that constrain the interpolatory results of a network to be contained within a physically admissible region. Quite recently, [13] proposed adding physical constraints to generative adversarial networks (GANs) also considering projection as we do, while stressing the interplay between scientific computing and machine learning; we refer the interested reader to their work for even more motivation for such approaches.…”

Section: Introductionmentioning

confidence: 99%

“…(a) output from[32] (b) µ = 103 (c) µ = 10 2 (d) µ = 10 1 (e) µ = 10 0(f) ground truth (g) µ = 10 −1 (h) µ = 10 −2 (i) µ = 10 −3 (j) µ = 10 −Comparison of the ground truth (f), the network result (a), and a number of quasistatic postprocessed solutions. Note how the stiffest zero length springs in (b) produce a result close to[32], and how the weaker zero-length springs allow the cloth to drift too far from the ground truth.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Coercing machine learning to output physically accurate results

Geng

Johnson

Fedkiw

2020

Journal of Computational Physics

View full text Add to dashboard Cite

Many machine/deep learning artificial neural networks are trained to simply be interpolation functions that map input variables to output values interpolated from the training data in a linear/nonlinear fashion. Even when the input/output pairs of the training data are physically accurate (e.g. the results of an experiment or numerical simulation), interpolated quantities can deviate quite far from being physically accurate. Although one could project the output of a network into a physically feasible region, such a postprocess is not captured by the energy function minimized when training the network; thus, the final projected result could incorrectly deviate quite far from the training data. We propose folding any such projection or postprocess directly into the network so that the final result is correctly compared to the training data by the energy function. Although we propose a general approach, we illustrate its efficacy on a specific convolutional neural network that takes in human pose parameters (joint rotations) and outputs a prediction of vertex positions representing a triangulated cloth mesh. While the original network outputs vertex positions with erroneously high stretching and compression energies, the new network trained with our physics prior remedies these issues producing highly improved results. there will be large errors inf (w, x T ) when compared to y T . On the other hand, although one could create a network with many degrees of freedom in order to capture y T =f (w, x T ) as accurately as desired, even exactly,f (w, x) could oscillate wildly and inaccurately when x is not equal to x T , i.e. overfitting. See, e.g. [14,15,16,17]. One needs to take great care when designing the network architecture in order to avoid underfitting while still allowing for enough regularization to also avoid overfitting. Likewise, the form of the energy function and nature of the numerical optimization techniques also need careful consideration. Some of the most popular methods include variants of BFGS [18,19,20] and a number of methods based on gradient descent [21,22] (see also [23] and the references therein) or interpreting gradient descent as a numerical approximation to an ordinary differential equation to be solved via various approaches motivated by order of accuracy [24,25] and adaptive time-stepping [26,27,28,29,30].Devising a network architecture with enough representative capability to alleviate underfitting while still being amenable to the regularization required to avoid overfitting, and subsequently applying numerical optimization techniques to an adequately designed energy in order to find reasonable parameters w is a quite difficult and mostly experimental endeavour. Thus, much of the progress made by the community emanates from the laborious creation of data sets that many researchers can consider in order to design network architectures and find suitable parameters w, see e.g. [31]. This is typically driven by a community (rather than an individual or group) effort, and state-of-the-art r...

show abstract

Machine learning of linear differential equations using Gaussian processes

Cited by 506 publications

References 31 publications

Explainable Machine Learning for Scientific Insights and Discoveries

Explainable Machine Learning for Scientific Insights and Discoveries

Multiscale Modeling Meets Machine Learning: What Can We Learn?

Coercing machine learning to output physically accurate results

Contact Info

Product

Resources

About