fPINNs: Fractional Physics-Informed Neural Networks

Abstract

Physics-informed neural networks (PINNs), introduced in [1], are effective in solving integerorder partial differential equations (PDEs) based on scattered and noisy data. PINNs employ standard feedforward neural networks (NNs) with the PDEs explicitly encoded into the NN using automatic differentiation, while the sum of the mean-squared PDE-residuals and the mean-squared error in initial/boundary conditions is minimized with respect to the NN parameters. Here we extend PINNs to fractional PINNs (fPINNs) to solve space-time fractional advection-diffusion equations (fractional ADEs), and we study systematically their convergence, hence explaining both of fPINNs and PINNs for first time. Specifically, we demonstrate their accuracy and effectiveness in solving multi-dimensional forward and inverse problems with forcing terms whose values are only known at randomly scattered spatio-temporal coordinates (black-box forcing terms). A novel element of the fPINNs is the hybrid approach that we introduce for constructing the residual in the loss function using both automatic differentiation for the integer-order operators and numerical discretization for the fractional operators. This approach bypasses the difficulties stemming from the fact that automatic differentiation is not applicable to fractional operators because the standard chain rule in integer calculus is not valid in fractional calculus. To discretize the fractional operators, we employ the Grünwald-Letnikov (GL) formula in one-dimensional fractional ADEs and the vector GL formula in conjunction with the directional fractional Laplacian in two-and three-dimensional fractional ADEs. We first consider the one-dimensional fractional Poisson equation and compare the convergence of the fPINNs against the finite difference method (FDM). We present the solution convergence using both the mean L 2 error as well as the standard deviation due to sensitivity to NN parameter initializations. Using different GL formulas we observe first-, second-, and third-order convergence rates for small size of training sets but the error saturates for larger training sets. We explain these results by analyzing the four sources of numerical errors due to discretization, sampling, NN approximation, and optimization. The total error decays monotonically (below 10 −5 for third order GL formula) but it saturates beyond that point due to the optimization error. We also analyze the relative balance between discretization and sampling errors and observe that the sampling size and the number of discretization points (auxiliary points) should be comparable to achieve the highest accuracy. As we increase the depth of the NN up to certain value, the mean error decreases and the standard deviation increases whereas the width has essentially no effect unless its value is either too small or too large. We next consider time-dependent fractional ADEs and compare white-box (WB) and black-box (BB) forcing. We observe that for the WB forcing, our results are similar to the aforementioned cases, however,...

Advancing Fusion with Machine Learning Research Needs Workshop Report

“…This can lead to discovery of new physics through direct use of data to determine analytic models that generate the observed physics, e.g. [6,42,53,60]. In this way parsimonious parameterized representations are discovered that minimize the mismatch between theory and data, but also potentially reveal hidden physics at play within the integrated multiphysics and engineering systems.…”

On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs

“…A series of works have shown the effectiveness of PINNs in one, two or three dimensional problems: fractional PDEs [30,36], stochastic differential equations [18,39], biomedical problems [33], and fluid mechanics [27]. Despite such remarkable success in these and related areas, PINNs lack theoretical justification.…”

Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations

“…PINNs can solve forward problems, where the solution of governing physical laws is inferred, as well as inverse problems, where unknown coefficients or even differential operators in the governing equations are identified. The PINNs has been applied extensively to solve various PDEs such as fractional PDEs [13,14], stochastic PDEs [11,12], with limited training data. Moreover, it has been successfully employed to solve many problems in computational and engineering science like, geostatistical modeling [36], cardiovascular systems [42][43][44], vortex-induced vibrations [45], high Mach number compressible flows [23], turbulent fluid flows [26,27], quantification of surface breaking cracks [19], uncertainty quantification [15,16], elastodynamics [28] etc.…”