Neural Networks with Cheap Differential Operators

Chen, Ricky T. Q.; Duvenaud, David

doi:10.48550/arxiv.1912.03579

Cited by 4 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The objective is to determine the dynamical system described by the equation ẋ(t) = F(x) from the known time series; here F : R n → R n is realized by a multilayer neural network [9][10][11]17,25]). The difference between the input x and the output F(x) is compared with ẋ(t) to generate an error e(t) ∈ R n .…”

Section: Algorithms For Adjusting the Weight Coefficients Of Neural O...mentioning

confidence: 99%

“…In recent years, an interesting idea has appeared to interpret a system of ordinary differential equations in the form of a suitable neural network (residual network) [9][10][11][12]. Precisely this interpretation is implemented in the present work: a system of differential equations (a system of so-called neural ODEs) is considered as a continuous analogue of some RNN [13][14][15][16][17].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Modeling of Chaotic Processes by Means of Antisymmetric Neural ODEs

Belozyorov

Dantsev

2022

JODEA

View full text Add to dashboard Cite

The main goal of this work is to construct an algorithm for modeling chaotic processes using special neural ODEs with antisymmetric matrices (antisymmetric neural ODEs) and power activation functions (PAFs). The central part of this algorithm is to design a neural ODEs architecture that would guarantee the generation of a stable limit cycle for a known time series. Then, one neuron is added to each equation of the created system until the approximating properties of this system satisfy the well-known Kolmogorov theorem on the approximation of a continuous function of many variables. In addition, as a result of such an addition of neurons, the cascade of bifurcations that allows generating a chaotic attractor from stable limit cycles is launched. We also consider the possibility of generating a homoclinic orbit whose bifurcations lead to the appearance of a chaotic attractor of another type. In conclusion, the conditions under which the found attractor adequately simulates the chaotic process are discussed. Examples are given.

show abstract

Section: Algorithms For Adjusting the Weight Coefficients Of Neural O...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Modeling of Chaotic Processes by Means of Antisymmetric Neural ODEs

Belozyorov

Dantsev

2022

JODEA

View full text Add to dashboard Cite

show abstract

“…Here, a challenge arises when a well-defined boundary condition is missing, and instead some additional data is available. The task then falls into the category of datadriven machine learning problems, which has attracted attention recently [14,15]. Also, a (potentially implicit) solution of the FP equation does not offer strategies to generate samples directly.…”

Section: Introductionmentioning

confidence: 99%

Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series

Paine,

Elfving,

Kyriienko

2021

Preprint

View full text Add to dashboard Cite

We propose a quantum algorithm for sampling from a solution of stochastic differential equations (SDEs). Using differentiable quantum circuits (DQCs) with a feature map encoding of latent variables, we represent the quantile function for an underlying probability distribution and extract samples as DQC expectation values. Using quantile mechanics we propagate the system in time, thereby allowing for time-series generation. We test the method by simulating the Ornstein-Uhlenbeck process and sampling at times different from the initial point, as required in financial analysis and dataset augmentation. Additionally, we analyse continuous quantum generative adversarial networks (qGANs), and show that they represent quantile functions with a modified (reordered) shape that impedes their efficient time-propagation. Our results shed light on the connection between quantum quantile mechanics (QQM) and qGANs for SDE-based distributions, and point the importance of differential constraints for model training, analogously with the recent success of physics informed neural networks.

show abstract

“…Fokker-Planck equation (FPE) is an important in studying properties of the dynamical systems, and has attracted a lot of attention in different fields. In recent years, FPE has become widespread in the machine learning community in the context of the important problems of density estimation [1] for neural ordinary differential equation (ODE) [2,3], generative models [4], etc. Consider a stochastic dynamical system which is described by stochastic differential equation (SDE) of the form 1 dx = f(x, t) dt + S(x, t) dβ, dβ dβ…”

Section: Introductionmentioning

confidence: 99%

“…Hence, to compute the value of ρ(x, t) at the specific point x = x, it is sufficient to find a preimage x 0 such that if it is used as an initial condition for (1), then we arrive to x. To find the preimage, we need to integrate the equation (1) backwards in time, and then to find the PDF value, we integrate a system of equations ( 1) and (3). Since we can evaluate the value of ρ(x, t) at any x, we can use the cross approximation method (CAM) [7,8,9] in the TTformat to recover a supposedly low-rank tensor from its samples.…”

Section: Introductionmentioning

confidence: 99%

Solution of the Fokker-Planck equation by cross approximation method in the tensor train format

Chertkov¹,

Oseledets²

2021

Preprint

View full text Add to dashboard Cite

We propose the novel numerical scheme for solution of the multidimensional Fokker-Planck equation, which is based on the Chebyshev interpolation and the spectral differentiation techniques as well as low rank tensor approximations, namely, the tensor train decomposition and the multidimensional cross approximation method, which in combination makes it possible to drastically reduce the number of degrees of freedom required to maintain accuracy as dimensionality increases. We demonstrate the effectiveness of the proposed approach on a number of multidimensional problems, including Ornstein-Uhlenbeck process and the dumbbell model. The developed computationally efficient solver can be used in a wide range of practically significant problems, including density estimation in machine learning applications.

show abstract

Neural Networks with Cheap Differential Operators

Cited by 4 publications

References 0 publications

Modeling of Chaotic Processes by Means of Antisymmetric Neural ODEs

Modeling of Chaotic Processes by Means of Antisymmetric Neural ODEs

Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series

Solution of the Fokker-Planck equation by cross approximation method in the tensor train format

Contact Info

Product

Resources

About