Exponential ergodicity for SDEs and McKean-Vlasov processes with Lévy noise

Liang, Mingjie; Majka, Mateusz B.; Wang, Jian

doi:10.48550/arxiv.1901.11125

Cited by 7 publications

(6 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are already many results derived for the ergodicity of McKean-Vlasov SDEs (i.e. DDSDEs) on R d , see for instances [8,10,13,14,15,25,27,38] among other references. It should be possible to extend most results to the reflecting setting.…”

Section: Log-harnack Inequality and Applicationsmentioning

confidence: 99%

Distribution Dependent Reflecting Stochastic Differential Equations

Wang¹

2021

Preprint

View full text Add to dashboard Cite

To characterize the Neumann problem for nonlinear Fokker-Planck equations, we investigate distribution dependent reflecting SDEs (DDRSDEs) in a domain. We first prove the well-posedness and establish functional inequalities for reflecting SDEs with singular drifts, then extend these results to DDRSDEs with singular or monotone coefficients, for which a general criterion deducing the well-posedness of DDRSDEs from that of reflecting SDEs is established. Moreover, three different types of exponential ergodicity are derived for DDRSDEs under dissipative, partially dissipative, and fully non-dissipative conditions respectively.

show abstract

Section: Log-harnack Inequality and Applicationsmentioning

confidence: 99%

Distribution Dependent Reflecting Stochastic Differential Equations

Wang¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, [PFF20] investigated the infinite width limits of fully connected networks initialized from a SαS distribution and proved heavy-tailed limiting distributions. On the other hand, heavy-tailed propagation of chaos results have been proven in theoretical probability [JMW08,LMW20]; however, their connection to SGD has not been yet established. We believe that (3.2) can be shown to hold under appropriate conditions, which we leave as future work.…”

Section: The Heavy-tailed Mean-field Regimementioning

confidence: 99%

Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks

Barsbey,

Sefidgaran,

Erdogdu

et al. 2021

Preprint

View full text Add to dashboard Cite

Neural network compression techniques have become increasingly popular as they can drastically reduce the storage and computation requirements for very large networks. Recent empirical studies have illustrated that even simple pruning strategies can be surprisingly effective, and several theoretical studies have shown that compressible networks (in specific senses) should achieve a low generalization error. Yet, a theoretical characterization of the underlying cause that makes the networks amenable to such simple compression schemes is still missing. In this study, we address this fundamental question and reveal that the dynamics of the training algorithm has a key role in obtaining such compressible networks. Focusing our attention on stochastic gradient descent (SGD), our main contribution is to link compressibility to two recently established properties of SGD: (i) as the network size goes to infinity, the system can converge to a mean-field limit, where the network weights behave independently, (ii) for a large step-size/batch-size ratio, the SGD iterates can converge to a heavy-tailed stationary distribution. In the case where these two phenomena occur simultaneously, we prove that the networks are guaranteed to be ' p -compressible', and the compression errors of different pruning techniques (magnitude, singular value, or node pruning) become arbitrarily small as the network size increases. We further prove generalization bounds adapted to our theoretical framework, which indeed confirm that the generalization error will be lower for more compressible networks. Our theory and numerical study on various neural networks show that large step-size/batch-size ratios introduce heavy-tails, which, in combination with overparametrization, result in compressibility. * Equal contribution.

show abstract

“…The coupling by reflection was applied in [4,5,6] to estimate the first eigenvalue on Riemannian manifolds as well as the spectral gap for elliptic diffusions, and was then developed in [21,22,24,7,25,16] for the exponential ergodicity of quasi-linear SPDEs, diffusion processes, and SDEs driven by Lévy noise. Unlike in the study of classical SDEs (or diffusion processes) for which we may let two marginal processes move together after the first meeting time (i.e.…”

Section: Introductionmentioning

confidence: 99%

Exponential Ergodicity for Non-Dissipative McKean-Vlasov SDEs

Wang¹

2021

Preprint

View full text Add to dashboard Cite

Under Lyapunov and monotone conditions, the exponential ergodicity in the induced Wasserstein quasi-distance is proved for a class of fully non-dissipative McKean-Vlasov SDEs, which strengthen some recent results established under dissipative conditions in long distance. Moreover, when the SDE is order-preserving, the exponential ergodicity is derived in the Wasserstein distance induced by one-dimensional increasing functions chosen according to the coefficients of the equation.

show abstract

Exponential ergodicity for SDEs and McKean-Vlasov processes with Lévy noise

Cited by 7 publications

References 50 publications

Distribution Dependent Reflecting Stochastic Differential Equations

Distribution Dependent Reflecting Stochastic Differential Equations

Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks

Exponential Ergodicity for Non-Dissipative McKean-Vlasov SDEs

Contact Info

Product

Resources

About