A Discussion on Solving Partial Differential Equations using Neural Networks

Dockhorn, Tim

doi:10.48550/arxiv.1904.07200

Cited by 24 publications

(49 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by the universal approximation theorems [3,4], recent studies have considered utilizing deep neural networks to solve PDEs [1,[5][6][7][8]. These studies present several different advantages that neural network based methods have over classical numerical methods.…”

Section: Introductionmentioning

confidence: 99%

Optimally weighted loss functions for solving PDEs with Neural Networks

Meer¹,

Oosterlee²,

Borovykh³

2020

Preprint

View full text Add to dashboard Cite

Recent works have shown that deep neural networks can be employed to solve partial differential equations, giving rise to the framework of physics informed neural networks [1]. We introduce a generalization for these methods that manifests as a scaling parameter which balances the relative importance of the different constraints imposed by partial differential equations. A mathematical motivation of these generalized methods is provided, which shows that for linear and well-posed partial differential equations, the functional form is convex. We then derive a choice for the scaling parameter that is optimal with respect to a measure of relative error. Because this optimal choice relies on having full knowledge of analytical solutions, we also propose a heuristic method to approximate this optimal choice. The proposed methods are compared numerically to the original methods on a variety of model partial differential equations, with the number of data points being updated adaptively. For several problems, including high-dimensional PDEs the proposed methods are shown to significantly enhance accuracy.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimally weighted loss functions for solving PDEs with Neural Networks

Meer¹,

Oosterlee²,

Borovykh³

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…the input space of (x, y) is thoroughly discretized, having ∆x = ∆y = 0.005. To generate an auxiliary task, we scale up the f (x, y), setting f aux (x, y) = −2π 2 sin(πx) sin(πy) as employed in [29]. We train our neural networks using fullbatch Adam with 0.005 learning rate for 50,000 epochs.…”

Section: Experiments and Resultsmentioning

confidence: 99%

Adversarial Multi-task Learning Enhanced Physics-informed Neural Networks for Solving Partial Differential Equations

Thanasutives,

Numao,

Fukui

2021

Preprint

View full text Add to dashboard Cite

Recently, researchers have utilized neural networks to accurately solve partial differential equations (PDEs), enabling the mesh-free method for scientific computation. Unfortunately, the network performance drops when encountering a high nonlinearity domain. To improve the generalizability, we introduce the novel approach of employing multi-task learning techniques, the uncertainty-weighting loss and the gradients surgery, in the context of learning PDE solutions. The multi-task scheme exploits the benefits of learning shared representations, controlled by cross-stitch modules, between multiple related PDEs, which are obtainable by varying the PDE parameterization coefficients, to generalize better on the original PDE. Encouraging the network pay closer attention to the high nonlinearity domain regions that are more challenging to learn, we also propose adversarial training for generating supplementary high-loss samples, similarly distributed to the original training distribution. In the experiments, our proposed methods are found to be effective and reduce the error on the unseen data points as compared to the previous approaches in various PDE examples, including high-dimensional stochastic PDEs.

show abstract

“…Some papers have worked on the application of such techniques to essential problems. For instance, [16] extensively explored the applications of solving the Poisson and the steady Navier-Stokes equations using NNs. [17] introduced a Physics Informed Neural Network (PINN) method to solve PDEs.…”

Section: Background and Motivationmentioning

confidence: 99%

A Swarm Variant for the Schrödinger Solver

Jivani¹,

Vaidya²,

Bhattacharya³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper introduces the application of the Exponentially Averaged Momentum Particle Swarm Optimization (EM-PSO) as a derivative-free optimizer for Neural Networks. It adopts PSO's major advantages such as search space exploration and higher robustness to local minima compared to gradient-descent optimizers such as Adam. Neural network based solvers endowed with gradient optimization are now being used to approximate solutions to Differential Equations. Here, we demonstrate the novelty of EM-PSO in approximating gradients and leveraging the property in solving the Schrödinger equation, for the Particle-in-a-Box problem. We also provide the optimal set of hyper-parameters supported by mathematical proofs, suited for our algorithm 1 .

show abstract

A Discussion on Solving Partial Differential Equations using Neural Networks

Cited by 24 publications

References 9 publications

Optimally weighted loss functions for solving PDEs with Neural Networks

Optimally weighted loss functions for solving PDEs with Neural Networks

Adversarial Multi-task Learning Enhanced Physics-informed Neural Networks for Solving Partial Differential Equations

A Swarm Variant for the Schrödinger Solver

Contact Info

Product

Resources

About