Neural Operator: Learning Maps Between Function Spaces

Abstract

The classical development of neural networks has primarily focused on learning mappings between finite dimensional Euclidean spaces or finite sets. We propose a generalization of neural networks tailored to learn operators mapping between infinite dimensional function spaces. We formulate the approximation of operators by composition of a class of linear integral operators and nonlinear activation functions, so that the composed operator can approximate complex nonlinear operators. We prove a universal approximation theorem for our construction. Furthermore, we introduce four classes of operator parameterizations: graph-based operators, low-rank operators, multipole graph-based operators, and Fourier operators and describe efficient algorithms for computing with each one. The proposed neural operators are resolution-invariant: they share the same network parameters between different discretizations of the underlying function spaces and can be used for zero-shot super-resolutions. Numerically, the proposed models show superior performance compared to existing machine learning based methodologies on Burgers' equation, Darcy flow, and the Navier-Stokes equation, while being several order of magnitude faster compared to conventional PDE solvers.

Learning Operators with Coupled Attention

“…al. [43], and thus also extends to Neural Operators via the correspondence between Neural Operators and (finite-dimensional) convolutional neural networks [46].…”

“…al. [46], where it is shown that a particular choice of neural operator architecture produces a DeepONet with an arbitrary trunk network and a branch network of a certain form. In particular, a Neural Operator layer has the form,…”

“…al. [46] that this architecture can be made to resemble a DeepONet under the following choices. First, set W ( ) = 0.…”

“…al. [46]. There, it was noted that if the kernel in the Neural Operator was (up to a linear transformation) of the form…”

“…Proof. The starting point of the proof is the following lemma, which gives justification for approximating operators on compact sets with finite dimensional subspaces as in [22,24,46]. The lemma follows immediately from the fact that for any compact subset U of a Banach space E and any > 0, there exists a finite dimensional subspace…”