Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs

Abstract

Stochastic partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. Neural Operators, generations of neural networks with capability of learning maps between infinite-dimensional spaces, are strong tools for solving parametric PDEs. However, they lack the ability to modeling SPDEs which usually have poor regularity 1 due to the driving noise. As the theory of regularity structure has achieved great successes in analyzing SPDEs and provides the concept model feature vectors that well-approximate SPDEs' solutions, we propose the Neural Operator with Regularity Structure (NORS) which incorporates the feature vectors for modeling dynamics driven by SPDEs. We conduct experiments on various of SPDEs including the dynamic Φ 4 1 model and the 2d stochastic Navier-Stokes equation, and the results demonstrate that the NORS is resolution-invariant, efficient, and achieves one order of magnitude lower error with a modest amount of data.

A Brief Survey on the Approximation Theory for Sequence Modelling

“…Other than deterministic model families, there are also results on modeling sequential data via (latent) neural controlled (stochastic) differential equations, such as hybrid architectures with GANs [44], universal neural operators for causality [30], and neural SPDE models motivated by mild solutions [40,71]. Applications include time series generation [60], irregular and long time series analysis [45,64], and online prediction [63].…”

Deep Latent Regularity Network for Modeling Stochastic Partial Differential Equations

“…Model feature vectors (Chevyrev, Gerasimovics, and Weber 2021) are the multi-dimensional generalization of path signature, i.e., from the temporal dimension to the spatial-temporal dimension. Both path signature and model feature vectors are used as features in applications for modelling spatial-temporal data, such as a solution of CDE (Morrill et al 2021) and SPDE (Chevyrev, Gerasimovics, and Weber 2021;Hu et al 2022) respectively. Because the computational complexity will dramatically increase when calculating high-order signatures, prior knowledge is needed on determining the degree of the features.…”