dNNsolve: an efficient NN-based PDE solver

Abstract

Neural Networks (NNs) can be used to solve Ordinary and Partial Differential Equations (ODEs and PDEs) by redefining the question as an optimization problem. The objective function to be optimized is the sum of the squares of the PDE to be solved and of the initial/boundary conditions. A feed forward NN is trained to minimise this loss function evaluated on a set of collocation points sampled from the domain where the problem is defined. A compact and smooth solution, that only depends on the weights of the trained NN, is then obtained. This approach is often referred to as 'PINN', from Physics Informed Neural Network [1,2]. Despite the success of the PINN approach in solving various classes of PDEs, an implementation of this idea that is capable of solving a large class of ODEs and PDEs with good accuracy and without the need to finely tune the hyperparameters of the network, is not available yet. In this paper, we introduce a new implementation of this concept -called dNNsolve -that makes use of dual Neural Networks to solve ODEs/PDEs. These include: i) sine and sigmoidal activation functions, that provide a more efficient basis to capture both secular and periodic patterns in the solutions; ii) a newly designed architecture, that makes it easy for the the NN to approximate the solution using the basis functions mentioned above. We show that dNNsolve is capable of solving a broad range of ODEs/PDEs in 1, 2 and 3 spacetime dimensions, without the need of hyperparameter fine-tuning.Preprint. Under review.

A scalable elliptic solver with task-based parallelism for the SpECTRE numerical relativity code

“…A possible avenue for a more drastic improvement of the elliptic solver algorithm is to replace the multigrid-Schwarz preconditioner, or parts of it, altogether. For example, recent developments in the field of physicsinformed neural networks (PINNs) suggest that hybrid strategies, combining a traditional linear solver with a PINN, can be very effective [68,69]. Hence, an intriguing prospect for accelerating elliptic solves in numerical relativity is to combine our Newton-Krylov algorithm with a PINN preconditioner, use the PINN as a smoother on multigrids, or use it to precondition Schwarz subdomain solves.…”

Qade: solving differential equations on quantum annealers

“…Methods based on machine learning techniques, first proposed in references [1][2][3][4], have gained increasing attention for their versatility in recent years [5][6][7][8][9][10][11][12][13][14][15][16][17]. At their core, these methods reformulate the task of solving differential equations as an optimization problem, for which the existing machine learning frameworks are designed [18][19][20][21][22][23][24][25][26][27].…”

Elvet -- a neural network-based differential equation and variational problem solver

“…A similar approach has been adapted in the deep Galerkin method using recurrent neural networks [28]. Recently the dNNsolve [29] method has been proposed to efficiently solve the differential equations with oscillatory nature, where a specially designed network with oscillatory and non-oscillatory components have been trained to approximate the solution 1 .…”