Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion

Abstract

Physics-informed neural networks (PINNs) have been proposed to learn the solution of partial differential equations (PDE). In PINNs, the residual form of the PDE of interest and its boundary conditions are lumped into a composite objective function as an unconstrained optimization problem, which is then used to train a deep feed-forward neural network.Here, we show that this specific way of formulating the objective function is the source of severe limitations in the PINN approach when applied to different kinds of PDEs. To address these limitations, we propose a versatile framework that can tackle both inverse and forward problems. The framework is adept at multi-fidelity data fusion and can seamlessly constrain the governing physics equations with proper initial and boundary conditions. The backbone of the proposed framework is a nonlinear, equality-constrained optimization problem formulation aimed at minimizing a loss functional, where an augmented Lagrangian method (ALM) is used to formally convert a constrained-optimization problem into an unconstrainedoptimization problem. We implement the ALM within a stochastic, gradient-descent type training algorithm in a way that scrupulously focuses on meeting the constraints without sacrificing other loss terms. Additionally, as a modification of the original residual layers, we propose lean residual layers in our neural network architecture to address the so-called vanishing-gradient problem. We demonstrate the efficacy and versatility of our physics-and equality-constrained deep-learning framework by applying it to learn the solutions of various multi-dimensional PDEs, including a nonlinear inverse problem from the hydrology field with multi-fidelity data fusion. The results produced with our proposed model match exact solutions very closely for all the cases considered.

Optimal control of PDEs using physics-informed neural networks

“…Yet, optimal control problems pose an additional challenge due to the possible scale difference between the PDE loss components which should ideally vanish, and the cost objective which might remain finite in the true optimal solution. Several recent studies try to address the issue of balancing different objectives when training PINNs, using either adaptive weighting strategies [43,44,45,46] or augmented Lagrangian methods [36,47]. In the present paper, however, our goal is to evaluate the feasibility and performance of the original PINN framework in solving optimal control problems.…”

Bayesian Physics-Informed Extreme Learning Machine for Forward and Inverse PDE Problems with Noisy Data

“…Recently, physics-informed machine learning, as a powerful tool, has attracted increasing attention to addressing those challenges [3]. In particular, physics-informed neural network (PINN), a general framework for solving both forward and inverse PDE problems, encodes the underlying physics laws into loss function to constrain the space of admissible solutions and makes predictions for unknown terms given limited data [4,5,6]. PINN has been successfully used for solving PDEs or complex PDE-based physics problems in various domains, such as materialogy [7], medical diagnosis [8] and hidden fluid mechanics [9], etc.…”

Critical Investigation of Failure Modes in Physics-informed Neural Networks

“…Section III presents the technical background on supervised and unsupervised learning approaches. In section IV we compare three objective/loss formulations, one of which is the recently introduced Physics and Equality Constrained Artificial Neural Networks(PECANNs) [21] through numerical experiments on two elliptic PDE problems and provide insights from these comparisons. Finally, section V concludes the paper with a discussion of our findings and future directions.…”