A multi-level procedure for enhancing accuracy of machine learning algorithms

Lye, Kjetil Olsen; Mishra, Siddhartha; Molinaro, Roberto

doi:10.1017/s0956792520000224

Cited by 31 publications

(31 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This could in turn benefit the accuracy and the convergence of the training process (see, e.g. [ 46 ] for the development of multilevel DNN training algorithms, albeit in another class of applications). In the context of multiscale networks, identifying the appropriate copy-number scalings that give rise to reduced models with simpler dynamics is a highly specialised task requiring careful theoretical analysis [ 16 ].…”

Section: Discussionmentioning

confidence: 99%

DeepCME: A deep learning framework for computing solution statistics of the chemical master equation

2021

View full text Add to dashboard Cite

Stochastic models of biomolecular reaction networks are commonly employed in systems and synthetic biology to study the effects of stochastic fluctuations emanating from reactions involving species with low copy-numbers. For such models, the Kolmogorov’s forward equation is called the chemical master equation (CME), and it is a fundamental system of linear ordinary differential equations (ODEs) that describes the evolution of the probability distribution of the random state-vector representing the copy-numbers of all the reacting species. The size of this system is given by the number of states that are accessible by the chemical system, and for most examples of interest this number is either very large or infinite. Moreover, approximations that reduce the size of the system by retaining only a finite number of important chemical states (e.g. those with non-negligible probability) result in high-dimensional ODE systems, even when the number of reacting species is small. Consequently, accurate numerical solution of the CME is very challenging, despite the linear nature of the underlying ODEs. One often resorts to estimating the solutions via computationally intensive stochastic simulations. The goal of the present paper is to develop a novel deep-learning approach for computing solution statistics of high-dimensional CMEs by reformulating the stochastic dynamics using Kolmogorov’s backward equation. The proposed method leverages superior approximation properties of Deep Neural Networks (DNNs) to reliably estimate expectations under the CME solution for several user-defined functions of the state-vector. This method is algorithmically based on reinforcement learning and it only requires a moderate number of stochastic simulations (in comparison to typical simulation-based approaches) to train the “policy function”. This allows not just the numerical approximation of various expectations for the CME solution but also of its sensitivities with respect to all the reaction network parameters (e.g. rate constants). We provide four examples to illustrate our methodology and provide several directions for future research.

show abstract

Section: Discussionmentioning

confidence: 99%

DeepCME: A deep learning framework for computing solution statistics of the chemical master equation

2021

View full text Add to dashboard Cite

show abstract

“…Furthermore, building a surrogate model can be especially useful in uncertainty quantification [69]. NNs can also aid classical methods in solving goal-oriented tasks [15,44]. In addition to the aforementioned research directions, further work has been done on fusing NNs with classical numerical methods to assist, for example, in model-order reduction [40,61].…”

Section: Related Workmentioning

confidence: 99%

Numerical Solution of the Parametric Diffusion Equation by Deep Neural Networks

et al. 2021

View full text Add to dashboard Cite

We perform a comprehensive numerical study of the effect of approximation-theoretical results for neural networks on practical learning problems in the context of numerical analysis. As the underlying model, we study the machine-learning-based solution of parametric partial differential equations. Here, approximation theory for fully-connected neural networks predicts that the performance of the model should depend only very mildly on the dimension of the parameter space and is determined by the intrinsic dimension of the solution manifold of the parametric partial differential equation. We use various methods to establish comparability between test-cases by minimizing the effect of the choice of test-cases on the optimization and sampling aspects of the learning problem. We find strong support for the hypothesis that approximation-theoretical effects heavily influence the practical behavior of learning problems in numerical analysis. Turning to practically more successful and modern architectures, at the end of this study we derive improved error bounds by focusing on convolutional neural networks.

show abstract

“…In contrast to data science applications, in this context, one can control the training data's accuracy and combine many relatively cheap low-fidelity samples with a few high-fidelity samples. The theoretical arguments and numerical evidence in [5] show that such a multi-level procedure can lower the generalisation error considerably. Becker et al [1] tackle optimal stopping problems for financial derivatives such as American or Bermudan options.…”

mentioning

confidence: 95%