Space-time error estimates for deep neural network approximations for differential equations

Abstract

Over the last few years deep artificial neural networks (DNNs) have very successfully been used in numerical simulations for a wide variety of computational problems including computer vision, image classification, speech recognition, natural language processing, as well as computational advertisement. In addition, it has recently been proposed to approximate solutions of partial differential equations (PDEs) by means of stochastic learning problems involving DNNs. There are now also a few rigorous mathematical results in the scientific literature which provide error estimates for such deep learning based approximation methods for PDEs. All of these articles provide spatial error estimates for neural network approximations for PDEs but do not provide error estimates for the entire space-time error for the 1 considered neural network approximations. It is the subject of the main result of this article to provide space-time error estimates for DNN approximations of Euler approximations of certain perturbed differential equations. Our proof of this result is based (i) on a certain artificial neural network (ANN) calculus and (ii) on ANN approximation results for products of the form r0, T s ˆRd Q pt, xq Þ Ñ tx P R d where T P p0, 8q, d P N, which we both develop within this article.

Efficient approximation of high-dimensional functions with neural networks

“…The following result is straight-forward from the definition. A formal proof can be found in Grohs et al [7].…”

“…Statements (i)-(ii) in the next proposition follow immediately from the definition. For (iii)-(iv), we refer to Grohs et al [7].…”

“…To do that, we first consider the square function sq : R → R, x → x 2 . It has been shown by different authors that the square function can be approximated with accuracy ε > 0 on the unit interval by a ReLU network φ ε satisfying P(φ ε ) = O(log 2 (ε −1 )); see [7,8,18,20]. This relies on realizing linear combinations of iterations of the sawtooth function with a neural network and establishing exponentially fast convergence in the number of iterations.…”

“…The rest of the paper is organized as follows. In Section 2, we establish the notation, recall basic facts from [7,12,17] about the concatenation and parallelization of neural networks and derive two simple consequences that are needed later in the paper. In Section 3, we analyze the approximability of catalogs with neural networks and derive first consequences for the corresponding catalog networks.…”

Approximations with deep neural networks in Sobolev time-space

“…In recent years, methods from deep learning have been applied to the numerical solution of partial differential equations with impressive results [2,3,5,8,10,13,15,16,17,20,21,24,26,27,34]. One key component of this success lies in the expressive power of neural networks, which constitute a parametrizes class of functions constructed by iterative compositions of affine mappings and pointwise application of a nonlinear activation function.…”