Solving Non-linear Kolmogorov Equations in Large Dimensions by Using Deep Learning: A Numerical Comparison of Discretization Schemes

Abstract

Non-linear partial differential Kolmogorov equations are successfully used to describe a wide range of time dependent phenomena, in natural sciences, engineering or even finance. For example, in physical systems, the Allen-Cahn equation describes pattern formation associated to phase transitions. In finance, instead, the Black-Scholes equation describes the evolution of the price of derivative investment instruments. Such modern applications often require to solve these equations in high-dimensional regimes in which classical approaches are ineffective. Recently, an interesting new approach based on deep learning has been introduced byby E, Han and Jentzen [1,2]. The main idea is to construct a deep network which is trained from the samples of discrete stochastic differential equations underlying Kolmogorov's equation. The network is able to approximate, numerically at least, the solutions of the Kolmogorov equation with polynomial complexity in whole spatial domains. In this contribution we study variants of the deep networks by using different discretizations schemes of the stochastic differential equation. We compare the performance of the associated networks, on benchmarked examples, and show that, for some discretization schemes, improvements in the accuracy are possible without affecting the observed computational complexity.

Phase transitions in the mini-batch size for sparse and dense two-layer neural networks

“…A deep neural network is a type of ML model, and when a deep network is fitted to data, this is referred to as deep learning [31]. Deep learning (DL) has shown very powerful empirical performance for solving very complex real-world problems in areas such as computer vision [32], natural language processing [33,34], speech recognition [35], recommendation systems [36], drug discovery [37], differential equations [38,39], and much more [40][41][42]. In simple words, DL can be seen a neural network [43], composed by many layers, that takes some data set D, input and targets, and learns the rules for forecasting new input data.…”

Large Independent Sets on Random d-Regular Graphs with Fixed Degree d

“…The MIS is important for applications in computer science, operations research, and engineering via such uses as graph coloring, assigning channels to the radio stations, register allocation in a compiler, artificial intelligence etc. [5][6][7][8].…”

Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems

“…If one wants to use the MC method for computing the global minimum, one can either run the algorithm at T = 0 or change slowly the temperature from an initial value to T = 0 : this is the so-called Simulated Annealing algorithm 14 . A key property that allows for a solid theory of MC is the so-called detailed balance condition that ensures the algorithm admits a limiting distribution at large times 15,16 .SGD [17][18][19] is a popular optimization algorithm used in the development of state-of-the-art machine learning 20 and deep learning models 21,22 , which have shown tremendous success in numerous fields, becoming indispensable tools for many advanced applications [23][24][25][26][27][28][29][30][31] . It is an extension of the gradient descent algorithm 32 that uses a subset of the training data to compute the gradient of the objective function at each iteration.…”

“…SGD [17][18][19] is a popular optimization algorithm used in the development of state-of-the-art machine learning 20 and deep learning models 21,22 , which have shown tremendous success in numerous fields, becoming indispensable tools for many advanced applications [23][24][25][26][27][28][29][30][31] . It is an extension of the gradient descent algorithm 32 that uses a subset of the training data to compute the gradient of the objective function at each iteration.…”