A Dual-Dimer method for training physics-constrained neural networks with minimax architecture

Liu, Dehao; Wang, Yan

doi:10.1016/j.neunet.2020.12.028

Cited by 46 publications

(13 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PINN methodology [109,146,191], many other variants were proposed, as the variational hp-VPINN, as well as conservative PINN (CPINN) [71]. Another approach is physics-constrained neural networks (PCNNs) [97,168,200]. while PINNs incorporate both the PDE and its initial/boundary conditions (soft BC) in the training loss function, PCNNs, are "data-free" NNs, i.e.…”

Section: What the Pinns Arementioning

confidence: 99%

Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next

et al. 2022

View full text Add to dashboard Cite

Physics-Informed Neural Networks (PINN) are neural networks (NNs) that encode model equations, like Partial Differential Equations (PDE), as a component of the neural network itself. PINNs are nowadays used to solve PDEs, fractional equations, integral-differential equations, and stochastic PDEs. This novel methodology has arisen as a multi-task learning framework in which a NN must fit observed data while reducing a PDE residual. This article provides a comprehensive review of the literature on PINNs: while the primary goal of the study was to characterize these networks and their related advantages and disadvantages. The review also attempts to incorporate publications on a broader range of collocation-based physics informed neural networks, which stars form the vanilla PINN, as well as many other variants, such as physics-constrained neural networks (PCNN), variational hp-VPINN, and conservative PINN (CPINN). The study indicates that most research has focused on customizing the PINN through different activation functions, gradient optimization techniques, neural network structures, and loss function structures. Despite the wide range of applications for which PINNs have been used, by demonstrating their ability to be more feasible in some contexts than classical numerical techniques like Finite Element Method (FEM), advancements are still possible, most notably theoretical issues that remain unresolved.

show abstract

Section: What the Pinns Arementioning

confidence: 99%

Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next

et al. 2022

View full text Add to dashboard Cite

show abstract

“…• physics-informed NNs (PINNs) (Raissi et al, 2019;Yang and Perdikaris, 2019;Meng et al, 2020) • physics-constrained neural networks (PCNNs) (Zhu et al, 2019;Sun et al, 2020a;Liu and Wang, 2021) PINNs incorporate both the PDE and its initial/boundary conditions (soft BC) in the training loss function, while physics-constrained NNs, i.e. "datafree" NNs, enforce the initial/boundary conditions (hard BC) via a custom NN architecture while embedding the PDE in the training loss.…”

Section: What Are Pinnsmentioning

confidence: 99%

Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next

Cuomo¹,

Cola²,

Giampaolo³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We should note that the authors in [25] only assume one type of boundary condition in their method. Liu and Wang [26] proposed to update these hyperparameters (i.e.,λ Ω , λ ∂Ω ) using gradient ascent while updating the parameters of their neural network model using a dual-dimer saddle point search algorithm. Following the approach presented in [26], McClenny and Braga-Neto [27] applied hyperparameters to individual training points in the domain and on the boundary.…”

Section: Physics-informed Neural Networkmentioning

confidence: 99%

“…Liu and Wang [26] proposed to update these hyperparameters (i.e.,λ Ω , λ ∂Ω ) using gradient ascent while updating the parameters of their neural network model using a dual-dimer saddle point search algorithm. Following the approach presented in [26], McClenny and Braga-Neto [27] applied hyperparameters to individual training points in the domain and on the boundary. We should note that reported results in [27] indicates that the author's approach marginally outperforms the empirical method proposed in [23].…”

Section: Physics-informed Neural Networkmentioning

confidence: 99%

Critical Investigation of Failure Modes in Physics-informed Neural Networks

Basir

Şenocak

2022

AIAA SCITECH 2022 Forum

View full text Add to dashboard Cite

In this paper, we demonstrate and investigate several challenges that stand in the way of tackling complex problems using physics-informed neural networks. In particular, we visualize the loss landscapes of trained models and perform sensitivity analysis of backpropagated gradients in the presence of physics. Our findings suggest that existing methods produce highly non-convex loss landscapes that are difficult to navigate. Furthermore, high-order PDEs contaminate the backpropagated gradients that may impede or prevent convergence. We then propose a novel method that bypasses the calculation of high-order PDE operators and mitigates the contamination of backpropagating gradients. In doing so, we reduce the dimension of the search space of our solution and facilitate learning problems with non-smooth solutions. Our formulation also provides a feedback mechanism that helps our model adaptively focus on complex regions of the domain that are difficult to learn. We then formulate an unconstrained dual problem by adapting the Lagrange multiplier method. We apply our method to solve several challenging benchmark problems governed by linear and non-linear PDEs.

show abstract

A Dual-Dimer method for training physics-constrained neural networks with minimax architecture

Cited by 46 publications

References 35 publications

Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next

Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next

Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next

Critical Investigation of Failure Modes in Physics-informed Neural Networks

Contact Info

Product

Resources

About