DGM: A deep learning algorithm for solving partial differential equations

Abstract

High-dimensional PDEs have been a longstanding computational challenge. We propose to solve highdimensional PDEs by approximating the solution with a deep neural network which is trained to satisfy the differential operator, initial condition, and boundary conditions. Our algorithm is meshfree, which is key since meshes become infeasible in higher dimensions. Instead of forming a mesh, the neural network is trained on batches of randomly sampled time and space points. The algorithm is tested on a class of high-dimensional free boundary PDEs, which we are able to accurately solve in up to 200 dimensions. The algorithm is also tested on a high-dimensional Hamilton-Jacobi-Bellman PDE and Burgers' equation. The deep learning algorithm approximates the general solution to the Burgers' equation for a continuum of different boundary conditions and physical conditions (which can be viewed as a high-dimensional space). We call the algorithm a "Deep Galerkin Method (DGM)" since it is similar in spirit to Galerkin methods, with the solution approximated by a neural network instead of a linear combination of basis functions. In addition, we prove a theorem regarding the approximation power of neural networks for a class of quasilinear parabolic PDEs. Deep learning and high-dimensional PDEsHigh-dimensional partial differential equations (PDEs) are used in physics, engineering, and finance. Their numerical solution has been a longstanding challenge. Finite difference methods become infeasible in higher dimensions due to the explosion in the number of grid points and the demand for reduced time step size. If there are d space dimensions and 1 time dimension, the mesh is of size O d+1 . This quickly becomes computationally intractable when the dimension d becomes even moderately large. We propose to solve high-dimensional PDEs using a meshfree deep learning algorithm. The method is similar in spirit to the Galerkin method, but with several key changes using ideas from machine learning. The Galerkin method is a widely-used computational method which seeks a reduced-form solution to a PDE as a linear combination of basis functions. The deep learning algorithm, or "Deep Galerkin Method" (DGM), uses a deep neural network instead of a linear combination of basis functions. The deep neural network is trained to satisfy the differential operator, initial condition, and boundary conditions using stochastic gradient descent at randomly sampled spatial points. By randomly sampling spatial points, we avoid the need to form a mesh (which is infeasible in higher dimensions) and instead convert the PDE problem into a machine learning problem.DGM is a natural merger of Galerkin methods and machine learning. The algorithm in principle is straightforward; see Section 2. Promising numerical results are presented later in Section 4 for a class *

D3M: A Deep Domain Decomposition Method for Partial Differential Equations

“…where K 2 > 0 is a constant, and the error bound between u k i and N k i is proved from Theorem 7.1 in [37].…”

“…Proof. In each subdomain, the convergence can be obtained from the Theorems 7.1 and 7.3 in [37]. With the Proposition 2, the sequence {N n i } n∈N is uniform bounded in n, and the rates of convergence to the solution u * are related to overlapping areas.…”

Deep Learning of Markov Model Based Machines for Determination of Better Treatment Option Decisions for Infertile Women

“…There were also successful methods in deep learning algorithms which score patients in ICU (Intensive Care Unit) for their severity and to predict mortality without using any model based assumptions in scoring systems (42) and for other medical applications, for example detection of worms through endoscopy (43), ophthalmology studies (44), cardiovascular studies (45), Parkinson's disease data (46), medical scoring systems (47). Deep learning procedures involved in various levels of abstraction for ranking system models can be found in (48,49), applications for mathematical models, parameter computations and stability of algorithms are found in (50)(51)(52)(53)(54)(55)(56).…”

Coercing machine learning to output physically accurate results

“…Since a number of authors have begun to consider the use of machine/deep learning for problems in traditional computational physics, see e.g. [1,2,3,4,5,6,7,8,9,10,11,12], we are motivated to consider methodologies that constrain the interpolatory results of a network to be contained within a physically admissible region. Quite recently, [13] proposed adding physical constraints to generative adversarial networks (GANs) also considering projection as we do, while stressing the interplay between scientific computing and machine learning; we refer the interested reader to their work for even more motivation for such approaches.…”