A physics-informed operator regression framework for extracting data-driven continuum models

Patel, Ravi G.; Trask, Nathaniel; Wood, Mitchell; Cyr, Eric C

doi:10.1016/j.cma.2020.113500

Cited by 72 publications

(27 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On this point, it is worth noted that DeepONet was based from the onset on the theorem of Chen & Chen [2], whereas the formulation of FNO was not theoretically justified originally, and the recent theoretical work covers only invariant kernels. There are also other methods for operator regression such as [28,29,30,31], but in this work we only consider DeepONet and FNO.…”

Section: Introductionmentioning

confidence: 99%

A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

Lu,

Meng,

Cai

et al. 2021

Preprint

View full text Add to dashboard Cite

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 [1], and its original architecture was based on the universal approximation theorem of Chen & Chen [2]. The second one, named Fourier Neural Operator or FNO, was published in 2020 [3], and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to [4], FNO in its continuous form can be viewed as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also endow DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform.We consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries and especially noisy data, the performance of FNO deteriorates greatly. For example, for the instability wave analysis with only 0.1% noise added to the input data, the error of FNO increases 10,000 times making it inappropriate for such important applications, while there is hardly any effect of such noise on the DeepONet. This behavior of FNO may be related to an inherent mapping instability between function spaces, and hence future work should address this important issue. We also compare theoretically the two neural operators and obtain similar error estimates

show abstract

Section: Introductionmentioning

confidence: 99%

A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

Lu,

Meng,

Cai

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This architecture reflects the nature of memoryless ODEs that dependent only on the immediately preceding state. A consequence is that evaluation of prediction model (19) requires an iteration over time steps. Thus the computational expense of this approach includes a linear scaling with N t , in contrast to [4].…”

Section: Time-stepping Regression Methodsmentioning

confidence: 99%

“…Motivated by this perturbation, we consider the use of residual neural networks [14] or ResNets based not only on their favorable training properties but also on their established connection to dynamical systems (see for instance [15,16,17,18,19]). Figure 1 presents a schematic of single ResNet layer.…”

Section: Residual Neural Networkmentioning

confidence: 99%

Reduced Basis Approximations of Parameterized Dynamical Partial Differential Equations via Neural Networks

Peter¹,

Beckwith²,

Cyr³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Projection-based reduced order models are effective at approximating parameter-dependent differential equations that are parametrically separable. When parametric separability is not satisfied, which occurs in both linear and nonlinear problems, projection-based methods fail to adequately reduce the computational complexity. Devising alternative reduced order models is crucial for obtaining efficient and accurate approximations to expensive high-fidelity models. In this work, we develop a time-stepping procedure for dynamical parameter-dependent problems, in which a neural-network is trained to propagate the coefficients of a reduced basis expansion. This results in an online stage with a computational cost independent of the size of the underlying problem. We demonstrate our method on several parabolic partial differential equations, including a problem that is not parametrically separable.

show abstract

“…Additionally, the approach is not a complete data-dependent one, and hence, cannot be made oblivious to the knowledge of the underlying PDE structure. Finally, the closest stream of work to the problem we investigate is represented by the "Neural Operators" [16,49,50,51,58]. Being a complete data-driven approach, the neural operators method aims at learning the operator map without having knowledge of the underlying PDEs.…”

Section: Introductionmentioning

confidence: 99%

Multiwavelet-based Operator Learning for Differential Equations

Gupta,

Xiao,

Bogdan

2021

Preprint

View full text Add to dashboard Cite

The solution of a partial differential equation can be obtained by computing the inverse operator map between the input and the solution space. Towards this end, we introduce a multiwavelet-based neural operator learning scheme that compresses the associated operator's kernel using fine-grained wavelets. By explicitly embedding the inverse multiwavelet filters, we learn the projection of the kernel onto fixed multiwavelet polynomial bases. The projected kernel is trained at multiple scales derived from using repeated computation of multiwavelet transform. This allows learning the complex dependencies at various scales and results in a resolution-independent scheme. Compare to the prior works, we exploit the fundamental properties of the operator's kernel which enable numerically efficient representation. We perform experiments on the Korteweg-de Vries (KdV) equation, Burgers' equation, Darcy Flow, and Navier-Stokes equation. Compared with the existing neural operator approaches, our model shows significantly higher accuracy and achieves state-of-the-art in a range of datasets. For the time-varying equations, the proposed method exhibits a (2X ´10X) improvement (0.0018 (0.0033) relative L2 error for Burgers' (KdV) equation). By learning the mappings between function spaces, the proposed method has the ability to find the solution of a high-resolution input after learning from lower-resolution data.

show abstract

A physics-informed operator regression framework for extracting data-driven continuum models

Cited by 72 publications

References 30 publications

A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

Reduced Basis Approximations of Parameterized Dynamical Partial Differential Equations via Neural Networks

Multiwavelet-based Operator Learning for Differential Equations

Contact Info

Product

Resources

About