On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs

Shin, Yeonjong

doi:10.4208/cicp.oa-2020-0193

Cited by 149 publications

(69 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our numerical results, we showed that hPINN with an augmented Lagrangian converges to a good solution, but there are currently few theoretical guarantees due to the nonlinear, nonconvex nature of this formulation. However, the convergence of stochastic optimization with the augmented Lagrangian method was analyzed in [61], and the convergence of PINN for certain linear PDEs was proved in [55]. In future work, we hope to theoretically analyze the convergence of hPINN based on these results.…”

Section: Discussionmentioning

confidence: 99%

Physics-Informed Neural Networks with Hard Constraints for Inverse Design

Lu¹,

Pestourie²,

Yao³

et al. 2021

SIAM J. Sci. Comput.

216

View full text Add to dashboard Cite

\bfA \bfb \bfs \bft \bfr \bfa \bfc \bft . Inverse design arises in a variety of areas in engineering such as acoustic, mechanics, thermal/electronic transport, electromagnetism, and optics. Topology optimization is an important form of inverse design, where one optimizes a designed geometry to achieve targeted properties parameterized by the materials at every point in a design region. This optimization is challenging, because it has a very high dimensionality and is usually constrained by partial differential equations (PDEs) and additional inequalities. Here, we propose a new deep learning method---physics-informed neural networks with hard constraints (hPINNs)---for solving topology optimization. hPINN leverages the recent development of PINNs for solving PDEs, and thus does not require a large dataset (generated by numerical PDE solvers) for training. However, all the constraints in PINNs are soft constraints, and hence we impose hard constraints by using the penalty method and the augmented Lagrangian method. We demonstrate the effectiveness of hPINN for a holography problem in optics and a fluid problem of Stokes flow. We achieve the same objective as conventional PDE-constrained optimization methods based on adjoint methods and numerical PDE solvers, but find that the design obtained from hPINN is often smoother for problems whose solution is not unique. Moreover, the implementation of inverse design with hPINN can be easier than that of conventional methods because it exploits the extensive deep-learning software infrastructure.\bfK \bfe \bfy \bfw \bfo \bfr \bfd \bfs . inverse design, topology optimization, partial differential equations, physicsinformed neural networks, penalty method, augmented Lagrangian method \bfA \bfM \bfS \bfs \bfu \bfb \bfj \bfe \bfc \bft \bfc \bfl \bfa \bfs \bfs \bfi fi\bfc \bfa \bft \bfi \bfo \bfn \bfs . 35R30, 65K10, 68T20 \bfD \bfO \bfI .

show abstract

Section: Discussionmentioning

confidence: 99%

Physics-Informed Neural Networks with Hard Constraints for Inverse Design

Lu¹,

Pestourie²,

Yao³

et al. 2021

SIAM J. Sci. Comput.

216

View full text Add to dashboard Cite

show abstract

“…Regardless of the potential of PINNs to solve PDEs, several studies reported their failures and limitations (Fuks and Tchelepi, 2020;Sun et al, 2020;Wang et al, 2021). Although there are some theoretical studies on the convergence and error analysis on PINNs (e.g., Mishra and Molinaro, 2022;Shin et al, 2020), theoretical understanding of PINNs is still in its infancy (Karniadakis et al, 2021). We summarize the advantages and disadvantages of PINNs compared to traditional numerical methods (e.g., finite difference, finite element, and finite volume methods) to potentially use the method to solve essential questions in hydrology, including large-scale forward and inverse modeling.…”

Section: Advantages and Disadvantages Of Pinnsmentioning

confidence: 99%

Forward and inverse modeling of water flow in unsaturated soils with discontinuous hydraulic conductivities using physics-informed neural networks with domain decomposition

Bandai

Ghezzehei

2022

Preprint

View full text Add to dashboard Cite

Abstract. Modeling water flow in unsaturated soils is vital for describing various hydrological and ecological phenomena. Soil water dynamics is described by well-established physical laws (Richardson-Richards equation (RRE)). Solving the RRE is difficult due to the inherent non-linearity of the processes, and various numerical methods have been proposed to solve the issue. However, applying the methods to practical situations is very challenging because they require well-defined initial and boundary conditions. Recent advances in machine learning and the growing availability of soil moisture data provide new opportunities for addressing the lingering challenges. Specifically, physics-informed machine learning allows taking advantage of both the known physics and data-driven modeling. Here, we present a physics-informed neural networks (PINNs) method that approximates the solution to the RRE using neural networks while concurrently matching available soil moisture data. Although the ability of PINNs to solve partial differential equations, including the RRE, has been demonstrated previously, its potential applications and limitations are not fully known. This study conducted a comprehensive analysis of PINNs and carefully tested the accuracy of the solutions by comparing them with analytical solutions and accepted traditional numerical solutions. We demonstrated that the solutions by PINNs with adaptive activation functions are comparable with those by traditional methods. Furthermore, while a single neural network (NN) is adequate to represent a homogeneous soil, we showed that soil moisture dynamics in layered soils with discontinuous hydraulic conductivities are correctly simulated by PINNs with domain decomposition (using separate NNs for each unique layer). A key advantage of PINNs is the absence of the strict requirement for precisely prescribed initial and boundary conditions. In addition, unlike traditional numerical methods, PINNs provide an inverse solution without repeatedly solving the forward problem. We demonstrated the application of these advantages by successfully simulating infiltration and redistribution constrained by sparse soil moisture measurements. As a free by-product, we gain knowledge of the water flux over the entire flow domain, including the unspecified upper and bottom boundary conditions. Nevertheless, there remain challenges that require further development. Chiefly, PINNs are sensitive to the initialization of NNs and are significantly slower than traditional numerical methods.

show abstract

“…PINNs are capable of obtaining good approximation accuracy given a sufficient data points and an expressive neural network architecture when the given differential equation is well-posed and has unique solution [22]. Shin et al [27] have explained why the output of PINNs converges to the solution of differential equations.…”

Section: Physics-informed Neural Networkmentioning

confidence: 99%

Identification and prediction of time-varying parameters of COVID-19 model: a data-driven deep learning approach

Long

Furati

2021

International Journal of Computer Mathematics

View full text Add to dashboard Cite

Data-driven deep learning provides efficient algorithms for parameter identification of epidemiology models. Unlike the constant parameters, the complexity of identifying time-varying parameters is largely increased. In this paper, a variant of physics-informed neural network is adopted to identify the time-varying parameters of the Susceptible-Infectious-Recovered-Deceased model for the spread of COVID-19 by fitting daily reported cases. The learned parameters are verified by utilizing an ordinary differential equation solver to compute the corresponding solutions of this compartmental model. The effective reproduction number based on these parameters is calculated. Long Short-Term Memory neural network is employed to predict the future weekly time-varying parameters. The numerical simulations demonstrate that PINN combined with LSTM yields accurate and effective results.

show abstract

On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs

Cited by 149 publications

References 31 publications

Physics-Informed Neural Networks with Hard Constraints for Inverse Design

Physics-Informed Neural Networks with Hard Constraints for Inverse Design

Forward and inverse modeling of water flow in unsaturated soils with discontinuous hydraulic conductivities using physics-informed neural networks with domain decomposition

Identification and prediction of time-varying parameters of COVID-19 model: a data-driven deep learning approach

Contact Info

Product

Resources

About