Galerkin Neural Networks: A Framework for Approximating Variational Equations with Error Control

Ainsworth, Mark; Dong, Justin

doi:10.48550/arxiv.2105.14094

Cited by 5 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The current approach adopts the probabilistic viewpoint with the aim of improving approximation, while the previously cited works generally focus on quantifying uncertainty. In a deterministic context several works have pursued other strategies to realize convergence in deep networks [He et al, 2018, Cyr et al, 2020, Adcock and Dexter, 2021, Fokina and Oseledets, 2019, Ainsworth and Dong, 2021. In the context of ML for reduced basis construction, several works have focused primarily on using either Gaussian processes and PCA [Guo and Hesthaven, 2018] or classical/variational autoencoders as replacements for PCA Carlberg, 2020, Lopez andAtzberger, 2020] in classical ROM schemes; this is distinct from the control volume type surrogates considered in which requires a reduced basis corresponding to a partition of space.…”

Section: Introductionmentioning

confidence: 99%

Probabilistic partition of unity networks: clustering based deep approximation

Trask,

Gulian,

Huang

et al. 2021

Preprint

View full text Add to dashboard Cite

Partition of unity networks (POU-Nets) have been shown capable of realizing algebraic convergence rates for regression and solution of PDEs, but require empirical tuning of training parameters. We enrich POU-Nets with a Gaussian noise model to obtain a probabilistic generalization amenable to gradient-based minimization of a maximum likelihood loss. The resulting architecture provides spatial representations of both noiseless and noisy data as Gaussian mixtures with closed form expressions for variance which provides an estimator of local error. The training process yields remarkably sharp partitions of input space based upon correlation of function values. This classification of training points is amenable to a hierarchical refinement strategy that significantly improves the localization of the regression, allowing for higher-order polynomial approximation to be utilized. The framework scales more favorably to large data sets as compared to Gaussian process regression and allows for spatially varying uncertainty, leveraging the expressive power of deep neural networks while bypassing expensive training associated with other probabilistic deep learning methods. Compared to standard deep neural networks, the framework demonstrates hp-convergence without the use of regularizers to tune the localization of partitions. We provide benchmarks quantifying performance in high/low-dimensions, demonstrating that convergence rates depend only on the latent dimension of data within high-dimensional space. Finally, we introduce a new open-source data set of PDE-based simulations of a semiconductor device and perform unsupervised extraction of a physically interpretable reduced-order basis.

show abstract

Section: Introductionmentioning

confidence: 99%

Probabilistic partition of unity networks: clustering based deep approximation

Trask,

Gulian,

Huang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We emphasize that the procedure we propose can, in principle, complement any operator regression technique that can furnish high-quality spatial functions, e.g., [17][18][19] . Our technique can also be seen in the context of several important methodologies developed recently combining deep learning methods with variational formulations of PDEs [20][21][22] .…”

Section: Machine-learning-based Spectral Methods For Partial Differen...mentioning

confidence: 99%

Machine-learning-based spectral methods for partial differential equations

Meuris

Qadeer

Stinis

2023

Sci Rep

View full text Add to dashboard Cite

Spectral methods are an important part of scientific computing’s arsenal for solving partial differential equations (PDEs). However, their applicability and effectiveness depend crucially on the choice of basis functions used to expand the solution of a PDE. The last decade has seen the emergence of deep learning as a strong contender in providing efficient representations of complex functions. In the current work, we present an approach for combining deep neural networks with spectral methods to solve PDEs. In particular, we use a deep learning technique known as the Deep Operator Network (DeepONet) to identify candidate functions on which to expand the solution of PDEs. We have devised an approach that uses the candidate functions provided by the DeepONet as a starting point to construct a set of functions that have the following properties: (1) they constitute a basis, (2) they are orthonormal, and (3) they are hierarchical, i.e., akin to Fourier series or orthogonal polynomials. We have exploited the favorable properties of our custom-made basis functions to both study their approximation capability and use them to expand the solution of linear and nonlinear time-dependent PDEs. The proposed approach advances the state of the art and versatility of spectral methods and, more generally, promotes the synergy between traditional scientific computing and machine learning.

show abstract

“…However, their physics-constrained loss can lead to discretization artifacts and requires a pressure gradient regularizer for high Reynolds numbers. Further learning-based approaches also exploit graph representations in terms of graph neural networks [Harsch andRiedelbauch 2021, Sanchez-Gonzalez et al 2020], graph convolutional networks [Gao, Zahr, and Wang 2021] as well as mesh representations [Pfaff et al 2021] or subspace representations Spiliopoulos 2018b, Ainsworth andDong 2021]. Unfortunately, graph neural networks cannot make use of the highly efficient implementations for convolutional operations on grids and thus usually come with higher computational complexity per node compared to grid based CNNs.…”

Section: Related Workmentioning

confidence: 99%

Spline-PINN: Approaching PDEs without Data Using Fast, Physics-Informed Hermite-Spline CNNs

Wandel

Weinmann

Neidlin

et al. 2022

AAAI

View full text Add to dashboard Cite

Partial Differential Equations (PDEs) are notoriously difficult to solve. In general, closed form solutions are not available and numerical approximation schemes are computationally expensive. In this paper, we propose to approach the solution of PDEs based on a novel technique that combines the advantages of two recently emerging machine learning based approaches. First, physics-informed neural networks (PINNs) learn continuous solutions of PDEs and can be trained with little to no ground truth data. However, PINNs do not generalize well to unseen domains. Second, convolutional neural networks provide fast inference and generalize but either require large amounts of training data or a physics-constrained loss based on finite differences that can lead to inaccuracies and discretization artifacts. We leverage the advantages of both of these approaches by using Hermite spline kernels in order to continuously interpolate a grid-based state representation that can be handled by a CNN. This allows for training without any precomputed training data using a physics-informed loss function only and provides fast, continuous solutions that generalize to unseen domains. We demonstrate the potential of our method at the examples of the incompressible Navier-Stokes equation and the damped wave equation. Our models are able to learn several intriguing phenomena such as Karman vortex streets, the Magnus effect, Doppler effect, interference patterns and wave reflections. Our quantitative assessment and an interactive real-time demo show that we are narrowing the gap in accuracy of unsupervised ML based methods to industrial solvers for computational fluid dynamics (CFD) while being orders of magnitude faster.

show abstract

Galerkin Neural Networks: A Framework for Approximating Variational Equations with Error Control

Cited by 5 publications

References 24 publications

Probabilistic partition of unity networks: clustering based deep approximation

Probabilistic partition of unity networks: clustering based deep approximation

Machine-learning-based spectral methods for partial differential equations

Spline-PINN: Approaching PDEs without Data Using Fast, Physics-Informed Hermite-Spline CNNs

Contact Info

Product

Resources

About