Learning Neural PDE Solvers with Convergence Guarantees

Hsieh, Jun-Ting; Zhao, Shengjia; Eismann, Stephan; Mirabella, Lucia; Ermon, Stefano

doi:10.48550/arxiv.1906.01200

Cited by 25 publications

(34 citation statements)

References 14 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, [19] attempted to discover the hidden physics model from data by learning differential operators. A fast, iterative PDE-solver was proposed by learning to modify each iteration of the existing solver [10]. A deep Backward Stochastic Differential Equation (BSDE) solver was proposed and investigated in [8,25] for solving high-dimensional parabolic PDEs by reformulating them using BSDE.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Sobolev Training for Physics Informed Neural Networks

Son¹,

Jang²,

Han³

et al. 2021

Preprint

View full text Add to dashboard Cite

Approximating the numerical solutions of Partial Differential Equ ations (PDEs) using neural networks is a promising application of deep learning. The smooth architecture of a fully connected neural network is appropriate for finding the solutions of PDEs; the corresponding loss function can also be intuitively designed and guarantees the convergence for various kinds of PDEs. However, the rate of convergence has been considered as a weakness of this approach. This paper introduces a novel loss function for the training of neural networks to find the solutions of PDEs, making the training substantially efficient. Inspired by the recent studies that incorporate derivative information for the training of neural networks, we develop a loss function that guides a neural network to reduce the error in the corresponding Sobolev space. Surprisingly, a simple modification of the loss function can make the training process similar to Sobolev Training although solving PDEs with neural networks is not a fully supervised learning task. We provide several theoretical justifications for such an approach for the viscous Burgers equation and the kinetic Fokker-Planck equation. We also present several simulation results, which show that compared with the traditional L 2 loss function, the proposed loss function guides the neural network to a significantly faster convergence. Moreover, we provide the empirical evidence that shows that the proposed loss function, together with the iterative sampling techniques, performs better in solving high dimensional PDEs.

show abstract

Section: Related Workmentioning

confidence: 99%

“…The High-dimensional Poisson equation. The Poisson equation serves as an example problem in the recent literature; see[10, 26, 28]. In this section, we provide empirical results to demonstrate that the proposed loss functions perform satisfactorily when equipped with iterative sampling for solving high-dimensional PDEs; see [23] for more information.…”

mentioning

confidence: 99%

Sobolev Training for Physics Informed Neural Networks

Son¹,

Jang²,

Han³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Related efforts have also attempted to quantify the generalization error of these methods for specific problems [4,18,19,20] and explain the convergence difficulties that PINNs face when solving some PDEs [15]. Although much recent effort has been dedicated to investigating the PINN framework for solving PDEs, other neural PDE solution techniques exist, notably techniques which learn iterators that are not solutions to PDEs themselves but rather provide a method of quickly computing such solutions [21].…”

Section: Related Workmentioning

confidence: 99%

Learning To Estimate Regions Of Attraction Of Autonomous Dynamical Systems Using Physics-Informed Neural Networks

Scharzenberger¹,

Hays²

2021

Preprint

View full text Add to dashboard Cite

When learning to perform motor tasks in a simulated environment, neural networks must be allowed to explore their action space to discover new potentially viable solutions. However, in an online learning scenario with physical hardware, this exploration must be constrained by relevant safety considerations in order to avoid damage to the agent's hardware and environment. We aim to address this problem by training a neural network, which we will refer to as a "safety network", to estimate the region of attraction (ROA) of a controlled autonomous dynamical system. This safety network can thereby be used to quantify the relative safety of proposed control actions and prevent the selection of damaging actions. Here we present our development of the safety network by training an artificial neural network (ANN) to represent the ROA of several autonomous dynamical system benchmark problems. The training of this network is predicated upon both Lyapunov theory and neural solutions to partial differential equations (PDEs). By learning to approximate the viscosity solution to a specially chosen PDE that contains the dynamics of the system of interest, the safety network learns to approximate a particular function, similar to a Lyapunov function, whose zero level set is the boundary of the ROA. We train our safety network to solve these PDEs in a semi-supervised manner following a modified version of the Physics Informed Neural Network (PINN) approach, utilizing a loss function that penalizes disagreement with the PDE's initial and boundary conditions, as well as non-zero residual and variational terms. In future work we intend to apply this technique to reinforcement learning agents during motor learning tasks.

show abstract

“…In other work, Schmitt et al [23] used evolutionary computation methods to optimize a GMG algorithm. By learning from supervised data and using a U-Net architecture, Hsieh et al [24] trained a convolutional network to improve a GMG algorithm for the structured Poisson problem. Katrutsa et al [25] formulated the two-level V-cyle problem as a deep neural network and, using this formulation, they optimized the interpolation operator for GMG and evaluated the method on 1D structured grids.…”

Section: Related Workmentioning

confidence: 99%

Optimization-Based Algebraic Multigrid Coarsening Using Reinforcement Learning

Taghibakhshi¹,

MacLachlan²,

Olson³

et al. 2021

Preprint

View full text Add to dashboard Cite

Large sparse linear systems of equations are ubiquitous in science and engineering, such as those arising from discretizations of partial differential equations. Algebraic multigrid (AMG) methods are one of the most common methods of solving such linear systems, with an extensive body of underlying mathematical theory. A system of linear equations defines a graph on the set of unknowns and each level of a multigrid solver requires the selection of an appropriate coarse graph along with restriction and interpolation operators that map to and from the coarse representation. The efficiency of the multigrid solver depends critically on this selection and many selection methods have been developed over the years. Recently, it has been demonstrated that it is possible to directly learn the AMG interpolation and restriction operators, given a coarse graph selection. In this paper, we consider the complementary problem of learning to coarsen graphs for a multigrid solver. We propose a method using a reinforcement learning (RL) agent based on graph neural networks (GNNs), which can learn to perform graph coarsening on small training graphs and then be applied to unstructured large graphs. We demonstrate that this method can produce better coarse graphs than existing algorithms, even as the graph size increases and other properties of the graph are varied. We also propose an efficient inference procedure for performing graph coarsening that results in linear time complexity in graph size.Preprint. Under review.

show abstract

Learning Neural PDE Solvers with Convergence Guarantees

Cited by 25 publications

References 14 publications

Sobolev Training for Physics Informed Neural Networks

Sobolev Training for Physics Informed Neural Networks

Learning To Estimate Regions Of Attraction Of Autonomous Dynamical Systems Using Physics-Informed Neural Networks

Optimization-Based Algebraic Multigrid Coarsening Using Reinforcement Learning

Contact Info

Product

Resources

About