Approximation of the invariant measure of stable SDEs by an Euler--Maruyama scheme

Abstract

We study approximations of the invariant measure of a stochastic differential equation (SDE) driven by an α-stable Lévy process (1 < α < 2) using the Euler-Maruyama (EM) scheme. By a discrete version of Duhamel's principle and Bismut's formula in Malliavin calculus, we show that the Wasserstein-1 distance of the invariant measures of the solution and the approximation is -up to a constant -bounded by η 2 α −1 where η is the step size of the EM scheme. An explicit calculation for Ornstein-Uhlenbeck α-stable process shows that the rate ηexample is given, and the relevant simulations are implemented. CONTENTS 1. Introduction 1.1. Motivation, contribution and method 1.2. Notation 1.3. Assumptions and main result 2. Ergodicity and moment estimates 3. Proof of Theorem 1.2 3.1. Auxiliary lemmas 3.2. Proof of Theorem 1.2 4. Malliavin calculus and the proof of Lemma 3.1 4.1. Jacobi flow associated with the SDE (1.1) 4.2. Bismut's formula 4.3. Time-change method for the SDE (1.1) 4.4. Proof of Lemma 3.1 5. Example and numerical simulations Appendix A. Proofs of Propositions 2.1, 2.2 and 2.3 and Lemma 2.5 Appendix B. Exact rate for the Ornstein-Uhlenbeck process References

Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions

“…• By combining our results with [6], we extend our bounds to the Euler-Maruyama discretization of (4) (that is of the form of ( 3)) and show that for small enough step-sizes the discrete-time process achieves almost identical stability bounds. Contrary to [14,18,27], our bounds do not rely on any topological regularity assumptions and they further do not contain a mutual information term.…”

“…Our results cover both the finite-time case, i.e., t < ∞ and the stationary case, i.e., t → ∞. We build upon recently introduced stochastic analysis tools for uniform-in-time Wasserstein error bounds for Euler-Maruyama discretization [6] to obtain a novel Wasserstein stability bound for two α-stable Lévy-driven SDEs. Our analysis relies on an additional pseudo-Lipschitz like condition for the underlying process and the dataset (Assumption 3) and careful adaption of the tools in [6] to our context (Lemma 18 and Theorem 12 in the Appendix) as well as additional analysis (Lemma 7) that allows us to characterize the dependence of our bounds on the tail-index α.…”

Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares

“…Thanks to the linearity of the drifts of these SDEs, the stationary distribution is achieved very quickly, with an exponential rate [52]. Hence, to ease our analysis, we will assume that we have two samples from the stationary distributions of (10) and (11), say θ and θ. In other words, we set our learning algorithm such that it gives a random sample from the stationary distribution of the SDE determined by the dataset, i.e., A cont ((X, y)) = θ, and A cont (( X, ŷ)) = θ, where A cont denotes the continuous-time heavy-tailed SGD algorithm.…”

“…Algorithmic stability via characteristic function. By setting y = ŷ = 0 and invoking Lemma 3, the characteristic functions of stationary distributions corresponding to the SDEs (10) and (11) are respectively given as follows:…”

“…Due to discretization error, the stationary distribution of (20) differs from the one of the corresponding SDE. Yet, very recently the difference between the two stationary distributions has shown to be of order η 2/α−1 (tight) in the Wasserstein-1 metric for α < 2[10], which gives a tolerable difference for small η.…”