Efficient approximation of high-dimensional functions with neural networks

Cheridito, Patrick; Jentzen, Arnulf; Rossmannek, Florian

doi:10.48550/arxiv.1912.04310

Cited by 5 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, KL-HMC takes about 4 mins compared to about 20 mins for B-PINN-HMC. However, we remark that the truncated KL expansion would suffer from the "curse of dimensionality" when approximating high dimensional functions, while deep neural networks are known to be efficient for high-dimensional function approximation [27].…”

Section: Results and Comparisonsmentioning

confidence: 99%

B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data

Yang,

Meng,

Karniadakis

2020

Preprint

View full text Add to dashboard Cite

We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINN posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

show abstract

Section: Results and Comparisonsmentioning

confidence: 99%

B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data

Yang,

Meng,

Karniadakis

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Next note that Cheridito et al [7,Proposition II.5] assures that there exists ∈ which satisfies for all x, y ∈ R I( 1 ) that R a ( ) ∈ C(R 2I( 1 ) , R 2O( 1 ) ), R a ( )(x, y) = (R a ( 1 )(x), R a ( 2 )(y)), and…”

Section: Sums Of Annsmentioning

confidence: 99%

High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations

Beneventano¹,

Cheridito²,

Jentzen³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper we develop a new machinery to study the capacity of artificial neural networks (ANNs) to approximate high-dimensional functions without suffering from the curse of dimensionality. Specifically, we introduce a concept which we refer to as approximation spaces of artificial neural networks and we present several tools to handle those spaces. Roughly speaking, approximation spaces consist of sequences of functions which can, in a suitable way, be approximated by ANNs without curse of dimensionality in the sense that the number of required ANN parameters to approximate a function of the sequence with an accuracy ε > 0 grows at most polynomially both in the reciprocal 1/ε of the required accuracy and in the dimension d ∈ N = {1, 2, 3, . . .} of the function. We show under suitable assumptions that these approximation spaces are closed under various operations including linear combinations, formations of limits, and infinite compositions. To illustrate the power of the machinery proposed in this paper, we employ the developed theory to prove that ANNs have the capacity to overcome the curse of dimensionality in the numerical approximation of certain first order transport partial differential equations (PDEs). Under suitable conditions we even prove that approximation spaces are closed under flows of first order transport PDEs.

show abstract

“…The two fractional errors shown are for three and four interpolation points in each of the dimensions, respectively. In contrast, NNs are known to mitigate the "curse of dimensionality" 44 .…”

Section: Order-of-magnitude Reduction In Training Datamentioning

confidence: 99%

Active learning of deep surrogates for PDEs: Application to metasurface design

Pestourie,

Mroueh,

Nguyen

et al. 2020

Preprint

View full text Add to dashboard Cite

Surrogate models for partial-differential equations are widely used in the design of metamaterials to rapidly evaluate the behavior of composable components. However, the training cost of accurate surrogates by machine learning can rapidly increase with the number of variables. For photonic-device models, we find that this training becomes especially challenging as design regions grow larger than the optical wavelength. We present an active learning algorithm that reduces the number of training points by more than an order of magnitude for a neural-network surrogate model of optical-surface components compared to random samples. Results show that the surrogate evaluation is over two orders of magnitude faster than a direct solve, and we demonstrate how this can be exploited to accelerate large-scale engineering optimization.

show abstract

Efficient approximation of high-dimensional functions with neural networks

Cited by 5 publications

References 17 publications

B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data

B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data

High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations

Active learning of deep surrogates for PDEs: Application to metasurface design

Contact Info

Product

Resources

About