Neural SDEs as Infinite-Dimensional GANs

Kidger, Patrick; Foster, James C.; Li, Xuechen; Oberhauser, Harald; Lyons, Terry

doi:10.48550/arxiv.2102.03657

Cited by 9 publications

(11 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This refines many universal approximation theorems in the literature. For instance, it refines [148] which concerns functions defined on the unit cube, it provides a quantitative version of [85], and it parallels the findings of [129] beyond the case where K is a differentiable sub-manifold of R n ; all while allowing for trainable activation functions. Note that in the special case where R m = R, K = [0, 1] n , and σ is of singular-ReLU type, we recover [130, Theorem 1].…”

Section: Static Case -Universal Approximation Into Qas Spacesmentioning

confidence: 95%

See 1 more Smart Citation

Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer

Acciaio¹,

Kratsios²,

Pammer³

2022

Preprint

View full text Add to dashboard Cite

A. We introduce a universal class of geometric deep learning models, called metric hypertransformers (MHTs), capable of approximating any adapted map F : X Z → Y Z with approximable complexity, where X ⊆ R d and Y is any suitable metric space, and X Z (resp. Y Z ) capture all discrete-time paths on X (resp. Y ). Suitable spaces Y include various (adapted) Wasserstein spaces, all Fréchet spaces admitting a Schauder basis, and a variety of Riemannian manifolds arising from information geometry. Even in the static case, where f : X → Y is a Hölder map, our results provide the first (quantitative) universal approximation theorem compatible with any such X and Y . Our universal approximation theorems are quantitative, and they depend on the regularity of F, the choice of activation function, the metric entropy and diameter of X , and on the regularity of the compact set of paths whereon the approximation is performed. Our guiding examples originate from mathematical finance. Notably, the MHT models introduced here are able to approximate a broad range of stochastic processes' kernels, including solutions to SDEs, many processes with arbitrarily long memory, and functions mapping sequential data to sequences of forward rate curves.

show abstract

Section: Static Case -Universal Approximation Into Qas Spacesmentioning

confidence: 95%

“…Other than direct path generation, neural networks have been employed as function approximators for drift and diffusions of the modelled SDE system, for robust and data-driven model selection mechanisms, as e.g. done in [7,42,54,146,39,85,138].…”

mentioning

confidence: 99%

Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer

Acciaio¹,

Kratsios²,

Pammer³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…which is approximated with the conditional GAN. The approximation of S t+∆t | S t is then obtained by inverting Equation (22). Since values of the process can get arbitrarily close to zero, the generator may output negative values very close to 0.…”

Section: Data Pre-processingmentioning

confidence: 99%

“…Another approach is to apply the 'neural ODEs' by Chen et al [21] on SDEs. Kidger et al [22] 'fit' SDEs to time series data, where the SDE coefficients are given by NNs. A GAN architecture is used here as well, where the solution to the SDE defines the output of the 'neural SDE'.…”

Section: Introductionmentioning

confidence: 99%

Monte Carlo Simulation of SDEs using GANs

Rhijn¹,

Oosterlee²,

Grzelak³

et al. 2021

Preprint

View full text Add to dashboard Cite

Generative adversarial networks (GANs) have shown promising results when applied on partial differential equations and financial time series generation. We investigate if GANs can also be used to approximate one-dimensional Itô stochastic differential equations (SDEs). We propose a scheme that approximates the path-wise conditional distribution of SDEs for large time steps. Standard GANs are only able to approximate processes in distribution, yielding a weak approximation to the SDE. A conditional GAN architecture is proposed that enables strong approximation. We inform the discriminator of this GAN with the map between the prior input to the generator and the corresponding output samples, i.e. we introduce a 'supervised GAN'. We compare the input-output map obtained with the standard GAN and supervised GAN and show experimentally that the standard GAN may fail to provide a path-wise approximation. The GAN is trained on a dataset obtained with exact simulation. The architecture was tested on geometric Brownian motion (GBM) and the Cox-Ingersoll-Ross (CIR) process. The supervised GAN outperformed the Euler and Milstein schemes in strong error on a discretisation with large time steps. It also outperformed the standard conditional GAN when approximating the conditional distribution. We also demonstrate how standard GANs may give rise to non-parsimonious input-output maps that are sensitive to perturbations, which motivates the need for constraints and regularisation on GAN generators.

show abstract

“…The task of solving random partial differential equations was recently addressed with deep learning using physics-informed neural networks [17][18][19]. Unsupervised generative modelling often requires using adversarial training [20,21] and generative adversarial network (GAN) architecture [22]. Moreover, in cases where the information about system parameters is limited, or exemplary datasets are generated from measurement and not known initial conditions, we arrive at the task of equation discovery [23][24][25][26][27].…”

Section: Introductionmentioning

confidence: 99%

Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series

Paine,

Elfving,

Kyriienko

2021

Preprint

View full text Add to dashboard Cite

We propose a quantum algorithm for sampling from a solution of stochastic differential equations (SDEs). Using differentiable quantum circuits (DQCs) with a feature map encoding of latent variables, we represent the quantile function for an underlying probability distribution and extract samples as DQC expectation values. Using quantile mechanics we propagate the system in time, thereby allowing for time-series generation. We test the method by simulating the Ornstein-Uhlenbeck process and sampling at times different from the initial point, as required in financial analysis and dataset augmentation. Additionally, we analyse continuous quantum generative adversarial networks (qGANs), and show that they represent quantile functions with a modified (reordered) shape that impedes their efficient time-propagation. Our results shed light on the connection between quantum quantile mechanics (QQM) and qGANs for SDE-based distributions, and point the importance of differential constraints for model training, analogously with the recent success of physics informed neural networks.

show abstract

Neural SDEs as Infinite-Dimensional GANs

Cited by 9 publications

References 0 publications

Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer

Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer

Monte Carlo Simulation of SDEs using GANs

Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series

Contact Info

Product

Resources

About