Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks

Wang, Sifan; Teng, Yujun; Perdikaris, Paris

doi:10.1137/20m1318043

Cited by 647 publications

(424 citation statements)

References 22 publications

Supporting

Mentioning

324

Contrasting

Unclassified

Order By: Relevance

“…However, we still lack the theoretical guarantees that such adaptive schemes converge, and there is no adaptive work for the parametric case. Finally, there is a need for PINN-specific training algorithms [110,112] and effective parallelization strategies [80], especially for massive GPUs.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Physics-informed neural networks for solving parametric magnetostatic problems

Beltrán-Pulido¹,

Bilionis²,

Aliprantis³

2022

Preprint

View full text Add to dashboard Cite

The optimal design of magnetic devices becomes intractable using current computational methods when the number of design parameters is high. The emerging physics-informed deep learning framework has the potential to alleviate this curse of dimensionality. The objective of this paper is to investigate the ability of physics-informed neural networks to learn the magnetic field response as a function of design parameters in the context of a two-dimensional (2-D) magnetostatic problem. Our approach is as follows. We derive the variational principle for 2-D parametric magnetostatic problems, and prove the existence and uniqueness of the solution that satisfies the equations of the governing physics, i.e., Maxwell's equations. We use a deep neural network (DNN) to represent the magnetic field as a function of space and a total of ten parameters that describe geometric features and operating point conditions. We train the DNN by minimizing the physics-informed loss function

show abstract

Section: Discussionmentioning

confidence: 99%

“…The third and last DNN architecture we consider is the modified residual neural network (ModResNet), proposed by [112]. The forward pass of a scalar value ModResNet with L hidden layers is defined recursively by:…”

Section: Modified Residual Neural Networkmentioning

confidence: 99%

Physics-informed neural networks for solving parametric magnetostatic problems

Beltrán-Pulido¹,

Bilionis²,

Aliprantis³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…where MSE F corresponds to the Euler equation ( 3) including the entropy conditions, MSE In f low corresponds to the inflow boundary conditions on primitive variables, MSE ∇ρ| D corresponds to the density gradient and D ⊂ Ω is a small region of the computational domain, MSE p * corresponds to the pressure data on the wall surface. We also employ dynamic weights [44] denoted as ω i , i = 1, 2, • • • in front of all the MSE terms in the loss function. Hereafter, for brevity, the loss function J is denoted without showing its dependence on the network parameters Θ.…”

Section: Expansion Wave Problemmentioning

confidence: 99%

Physics-informed neural networks for inverse problems in supersonic flows

Jagtap,

Mao,

Adams

et al. 2022

Preprint

View full text Add to dashboard Cite

Accurate solutions to inverse supersonic compressible flow problems are often required for designing specialized aerospace vehicles. In particular, we consider the problem where we have data available for density gradients from Schlieren photography as well as data at the inflow and part of wall boundaries. These inverse problems are notoriously difficult and traditional methods may not be adequate to solve such ill-posed inverse problems. To this end, we employ the physics-informed neural networks (PINNs) and its extended version, extended PINNs (XPINNs), where domain decomposition allows to deploy locally powerful neural networks in each subdomain, which can provide additional expressivity in subdomains, where a complex solution is expected. Apart from the governing compressible Euler equations, we also enforce the entropy conditions in order to obtain viscosity solutions. Moreover, we enforce positivity conditions on density and pressure. We consider inverse problems involving two-dimensional expansion waves, two-dimensional oblique and bow shock waves. We compare solutions obtained by PINNs and XPINNs and invoke some theoretical results that can be used to decide on the generalization errors of the two methods.

show abstract

“…By trial and error, we found suitable weights (Wang, et al, 2020) that allow for good convergence of the loss function. The values are seen in Eq.…”

Section: Homogeneous Media Case Using Xpinnmentioning

confidence: 99%

Extended Physics-Informed Neural Networks for Solving Fluid Flow Problems in Highly Heterogeneous Media

Alhubail

Alsinan

et al. 2022

Day 3 Wed, February 23, 2022

View full text Add to dashboard Cite

Utilization of neural networks to solve physical problems has been receiving wide attention recently. These neural networks are commonly named physics-informed neural network (PINN), in which the physics are employed through the governing partial differential equations (PDEs). Traditional PINNs suffer from unstable performance when dealing with flow problems in highly heterogeneous domains. This work presents the applicability of the extended PINN (XPINN) method in solving heterogeneous problems. XPINN can create a full solution model to the solution of the governing PDEs by training the neural network on the PDEs and its constraints such as boundary and initial conditions, and known solution points. The heterogeneous problem is solved by performing domain decomposition, which divides the original heterogeneous domain into various homogeneous sub-domains. Each sub-domain incorporates its own PINN structure. The different PINNs are connected through interface conditions, allowing for information to communicate across the interfaces. These conditions include pressure and flux continuities. Various heterogeneous scenarios are implemented in this study to investigate the robustness of the proposed method. We demonstrate the accuracy of the XPINN model by comparing it with the ground truth solved from high-fidelity simulations. Results show a good match in terms of pressure and velocity with errors of less than 1%. Different interface conditions were tested, and it was observed that without the inclusion of pressure and flux continuities, the solver does not converge to the solution of interest. Sensitivity analysis was performed to explore the effects of the neural network architecture, the weights given to each loss term, and the number of training iterations. Results show that wide and shallow networks performed well due to avoiding the gradient vanishing issue that comes with deeper networks. In addition, balanced weights produced better accuracy in general. Moreover, more training iterations improved the accuracy of the results but at lower rates in later training stages. This paper presents XPINN to solve fluid flow in heterogeneous media. We demonstrate the robustness and accuracy of the proposed XPINN model by comparing it with the ground truth solutions in multiple heterogeneous cases. The model shows good potential and can be readily implemented in reservoir characterization workflow.

show abstract

Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks

Cited by 647 publications

References 22 publications

Physics-informed neural networks for solving parametric magnetostatic problems

Physics-informed neural networks for solving parametric magnetostatic problems

Physics-informed neural networks for inverse problems in supersonic flows

Extended Physics-Informed Neural Networks for Solving Fluid Flow Problems in Highly Heterogeneous Media

Contact Info

Product

Resources

About