Neural Ordinary Differential Equations

Chen, Ricky T. Q.; Rubanova, Yulia; Bettencourt, Jesse; Duvenaud, David

doi:10.48550/arxiv.1806.07366

Cited by 175 publications

(318 citation statements)

References 0 publications

Supporting

Mentioning

315

Contrasting

Unclassified

Order By: Relevance

“…We first estimate the drift g from the observation data using the variational formulation for the stationary Fokker-Planck equation ( 9) through relative entropy rate. Suppose that the stationary probability density p g (x) is given and we search for a estimator of the drift g(x) by minimizing the relative entropy rate (6). Introducing a Lagrange multiplier function ψ(x), we may derive the drift g(x) from the following Lagrange functional…”

Section: Variational Formulationmentioning

confidence: 99%

See 1 more Smart Citation

Nonparametric inference of stochastic differential equations based on the relative entropy rate

Dai¹,

Duan²,

Hu³

et al. 2021

Preprint

View full text Add to dashboard Cite

The information detection of complex systems from data is currently undergoing a revolution, driven by the emergence of big data and machine learning methodology. Discovering governing equations and quantifying dynamical properties of complex systems are among central challenges. In this work, we devise a nonparametric approach to learn the relative entropy rate from observations of stochastic differential equations with different drift functions. The estimator corresponding to the relative entropy rate then is presented via the Gaussian process kernel theory. Meanwhile, this approach enables to extract the governing equations. We illustrate our approach in several examples. Numerical experiments show the proposed approach performs well for rational drift functions, not only polynomial drift functions.

show abstract

Section: Variational Formulationmentioning

confidence: 99%

“…There exist many different forms in regard to data-driven methods, such as parametric and nonparametric approaches [6][7][8][9][10]. The recent sparse identification of nonlinear dynamics method, which proposed by Brunton and co-workers [11], is a scriptures of parametric approaches.…”

Section: Introductionmentioning

confidence: 99%

Nonparametric inference of stochastic differential equations based on the relative entropy rate

Dai¹,

Duan²,

Hu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We do not, however, have to resign ourselves to treating them as black boxes. Driven in part by the desire to derive physical intuition from deep methods, interrogation and interpretation methods for deep learning are an active and important area of research (e.g., Doshi-Velez & Kim 2017;Slavin Ross et al 2017;Chen et al 2018;Rudin 2019;Winkler et al 2019;Wu et al 2020;Yu et al 2019).…”

Section: Introductionmentioning

confidence: 99%

The Importance of Being Interpretable: Toward An Understandable Machine Learning Encoder for Galaxy Cluster Cosmology

Ntampaka,

Vikhlinin

2021

Preprint

View full text Add to dashboard Cite

We present a deep machine learning (ML) approach to constraining cosmological parameters with multi-wavelength observations of galaxy clusters. The ML approach has two components: an encoder that builds a compressed representation of each galaxy cluster and a flexible CNN to estimate the cosmological model from a cluster sample. It is trained and tested on simulated cluster catalogs built from the Magneticum simulations. From the simulated catalogs, the ML method estimates the amplitude of matter fluctuations, σ 8 , at approximately the expected theoretical limit. More importantly, the deep ML approach can be interpreted. We lay out three schemes for interpreting the ML technique: a leave-one-out method for assessing cluster importance, an average saliency for evaluating feature importance, and correlations in the terse layer for understanding whether an ML technique can be safely applied to observational data. These interpretation schemes led to the discovery of a previously unknown self-calibration mode for flux-and volume-limited cluster surveys. We describe this new mode, which uses the amplitude and peak of the cluster mass PDF as anchors for mass calibration. We introduce the term "overspecialized" to describe a common pitfall in astronomical applications of machine learning in which the ML method learns simulation-specific details, and we show how a carefully constructed architecture can be used to check for this source of systematic error.

show abstract

“…Recent years have seen a resurgence of numerical methods based on (deep) artificial neural networks (ANNs) for solving ordinary (ODEs) and partial differential equations (PDEs) (Budkina et al, 2016;Chen et al, 2018;E et al, 2017;E and Yu, 2018;Khodayi-Mehr and Zavlanos, 2020;Long et al, 2019Long et al, , 2018Mall and Chakraverty, 2016;Raissi et al, 2019;Sirignano and Spiliopoulos, 2018;Yadav et al, 2015). These so-called neural solvers revisit an idea with origins more than 20 years ago (Aarts and Van Der Veer, 2001;Dissanayake and Phan-Thien, 1994;Lagaris et al, 1998Lagaris et al, , 2000Lee and Kang, 1990;Meade Jr and Fernandez, 1994) in the new light of the ongoing advances in machine learning (ML) technologies, the availability of deep learning software (Abadi et al, 2016;Al-Rfou et al, 2016;Haghighat and Juanes, 2021;Lu et al, 2021;Paszke et al, 2017), and the capabilities of modern computing hardware (Jouppi et al, 2018;LeCun, 2019).…”

Section: Introductionmentioning

confidence: 99%

A Neural Solver for Variational Problems on CAD Geometries with Application to Electric Machine Simulation

von Tresckow,

Kurz,

De Gersem

et al. 2021

Preprint

View full text Add to dashboard Cite

This work presents a deep learning-based framework for the solution of partial differential equations on complex computational domains described with computer-aided design tools. To account for the underlying distribution of the training data caused by spline-based projections from the reference to the physical domain, a variational neural solver equipped with an importance sampling scheme is developed, such that the loss function based on the discretized energy functional obtained after the weak formulation is modified according to the sample distribution. To tackle multi-patch domains possibly leading to solution discontinuities, the variational neural solver is additionally combined with a domain decomposition approach based on the Discontinuous Galerkin formulation. The proposed neural solver is verified on a toy problem and then applied to a real-world engineering test case, namely that of electric machine simulation. The numerical results show clearly that the neural solver produces physics-conforming solutions of significantly improved accuracy.

show abstract

Neural Ordinary Differential Equations

Cited by 175 publications

References 0 publications

Nonparametric inference of stochastic differential equations based on the relative entropy rate

Nonparametric inference of stochastic differential equations based on the relative entropy rate

The Importance of Being Interpretable: Toward An Understandable Machine Learning Encoder for Galaxy Cluster Cosmology

A Neural Solver for Variational Problems on CAD Geometries with Application to Electric Machine Simulation

Contact Info

Product

Resources

About