Three ways to solve partial differential equations with neural networks — A review

Blechschmidt, Jan; Ernst, Oliver G.

doi:10.1002/gamm.202100006

Cited by 138 publications

(73 citation statements)

References 138 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Various review papers involving PINN have been published. About the potentials, limitations and applications for forward and inverse problems (Karniadakis et al, 2021) or a comparison with other ML techniques (Blechschmidt and Ernst, 2021). An introductory course on PINNs that covers the fundamentals of Machine Learning and Neural Networks can be found from Kollmannsberger et al (2021).…”

Section: What Are Pinnsmentioning

confidence: 99%

See 1 more Smart Citation

Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next

Cuomo¹,

Cola²,

Giampaolo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Physics-Informed Neural Networks (PINN) are neural networks (NNs) that encode model equations, like Partial Differential Equations (PDE), as a component of the neural network itself. PINNs are nowadays used to solve PDEs, fractional equations, and integral-differential equations. This novel methodology has arisen as a multi-task learning framework in which a NN must fit observed data while reducing a PDE residual. This article provides a comprehensive review of the literature on PINNs: while the primary goal of the study was to characterize these networks and their related advantages and disadvantages, the

show abstract

Section: What Are Pinnsmentioning

confidence: 99%

“…Unfortunately, dealing with such high dimensional-complex systems are not exempt from the course of dimensionality, which Bellman first described in the context of optimal control problems (Bellman, 1966). However, machine learning-based algorithms are promising for solving PDEs (Blechschmidt and Ernst, 2021).…”

Section: Introductionmentioning

confidence: 99%

Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next

Cuomo¹,

Cola²,

Giampaolo³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…While many algorithms can be interpreted as solving an optimization problems or fixed-point computations and can therefore be improved with amortized optimization, it is also fruitful to use learning to improve algorithms that have nothing to do with optimization. Some key starting references in this space include data-driven algorithm design (Balcan, 2020), learning to prune (Alabi et al, 2019), learning solutions to differential equations Poli et al, 2020;Karniadakis et al, 2021;Kovachki et al, 2021;Chen et al, 2021b;Blechschmidt and Ernst, 2021;Marwah et al, 2021;Berto et al, 2021) learning simulators for physics (Grzeszczuk et al, 1998;Ladickỳ et al, 2015;He et al, 2019;Sanchez-Gonzalez et al, 2020;Wiewel et al, 2019;Usman et al, 2021;Vinuesa and Brunton, 2021), and learning for symbolic math (Lample and Charton, 2019;Charton, 2021;Drori et al, 2021;d'Ascoli et al, 2022) Salimans andHo (2022) progressively amortizes a sampling process for diffusion models.…”

Section: Learning-augmented and Amortized Algorithms Beyond Optimizationmentioning

confidence: 99%

Tutorial on amortized optimization for learning to optimize over continuous domains

Amos¹

2022

Preprint

View full text Add to dashboard Cite

Optimization is a ubiquitous modeling tool and is often deployed in settings which repeatedly solve similar instances of the same problem. Amortized optimization methods use learning to predict the solutions to problems in these settings. This leverages the shared structure between similar problem instances. In this tutorial, we will discuss the key design choices behind amortized optimization, roughly categorizing 1) models into fully-amortized and semi-amortized approaches, and 2) learning methods into regression-based and objectivebased. We then view existing applications through these foundations to draw connections between them, including for manifold optimization, variational inference, sparse coding, meta-learning, control, reinforcement learning, convex optimization, and deep equilibrium networks. This framing enables us easily see, for example, that the amortized inference in variational autoencoders is conceptually identical to value gradients in control and reinforcement learning as they both use fully-amortized models with an objective-based loss.

show abstract

“…Methods of machine learning were shown to be particularly effective for identifying the principal properties of the phenomena (for example, physical, economic, or social), which are stochastic by nature or contain some hidden parameters [1,2]. ML is also successfully used to solve complex problems of computational mathematics, for example, for simulation of dynamical systems [3], solution of ordinary, partial, or stochastic differential equations [4][5][6].…”

Section: Introductionmentioning

confidence: 99%

Acceleration of Global Optimization Algorithm by Detecting Local Extrema Based on Machine Learning

Barkalov

Lebedev

Kozinov

2021

Entropy

View full text Add to dashboard Cite

This paper features the study of global optimization problems and numerical methods of their solution. Such problems are computationally expensive since the objective function can be multi-extremal, nondifferentiable, and, as a rule, given in the form of a “black box”. This study used a deterministic algorithm for finding the global extremum. This algorithm is based neither on the concept of multistart, nor nature-inspired algorithms. The article provides computational rules of the one-dimensional algorithm and the nested optimization scheme which could be applied for solving multidimensional problems. Please note that the solution complexity of global optimization problems essentially depends on the presence of multiple local extrema. In this paper, we apply machine learning methods to identify regions of attraction of local minima. The use of local optimization algorithms in the selected regions can significantly accelerate the convergence of global search as it could reduce the number of search trials in the vicinity of local minima. The results of computational experiments carried out on several hundred global optimization problems of different dimensionalities presented in the paper confirm the effect of accelerated convergence (in terms of the number of search trials required to solve a problem with a given accuracy).

show abstract

Three ways to solve partial differential equations with neural networks — A review

Cited by 138 publications

References 138 publications

Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next

Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next

Tutorial on amortized optimization for learning to optimize over continuous domains

Acceleration of Global Optimization Algorithm by Detecting Local Extrema Based on Machine Learning

Contact Info

Product

Resources

About