Stochastic Approximations and Differential Inclusions

Benaı̈m, Michel; Hofbauer, Josef; Sorin, Sylvain

doi:10.1137/s0363012904439301

Cited by 253 publications

(535 citation statements)

References 21 publications

(15 reference statements)

Supporting

Mentioning

512

Contrasting

Order By: Relevance

“…Furthermore some of the assumptions which are typically used are challenging to verify. In this work we expand the asymptotic pseudo-trajectory approach of Benaïm, Hofbauer and Sorin [5] to asynchronous stochastic approximations with set-valued mean fields. We incorporate the asynchronicity into the mean field to give a differential inclusion which will characterise the limiting behaviour of the associated learning process.…”

mentioning

confidence: 99%

“…Commonly known as the ODE method of stochastic approximation, originally proposed by Ljung [20], this technique has been extended by numerous authors, for example Benaïm [3], Benaïm, Hofbauer and Sorin [5], Borkar [11], Kushner and Clark [16] and Kushner and Yin [17,18]. In particular Benaïm, Hofbauer and Sorin [5] have developed the approach so that under some weak criteria {x n } n∈N can be updated via a set-valued mean field, F (·).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Asynchronous Stochastic Approximation with Differential Inclusions

Perkins

Leslie

2012

Stochastic Systems

View full text Add to dashboard Cite

The asymptotic pseudo-trajectory approach to stochastic approximation of Benaïm, Hofbauer and Sorin is extended for asynchronous stochastic approximations with a set-valued mean field. The asynchronicity of the process is incorporated into the mean field to produce convergence results which remain similar to those of an equivalent synchronous process. In addition, this allows many of the restrictive assumptions previously associated with asynchronous stochastic approximation to be removed. The framework is extended for a coupled asynchronous stochastic approximation process with set-valued mean fields. Two-timescales arguments are used here in a similar manner to the original work in this area by Borkar. The applicability of this approach is demonstrated through learning in a Markov decision process.

show abstract

mentioning

confidence: 99%

mentioning

confidence: 99%

Asynchronous Stochastic Approximation with Differential Inclusions

Perkins

Leslie

2012

Stochastic Systems

View full text Add to dashboard Cite

show abstract

“…The proof of the theorem follows the ideas of [17], [23]. In fact, when Θ is restricted to a single value, we obtain again the result of [17].…”

Section: Sketch Of Proofmentioning

confidence: 52%

Mean Field Approximation of Uncertain Stochastic Models

Bortolussi

Gast

2016

2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)

View full text Add to dashboard Cite

Abstract-We consider stochastic models in presence of uncertainty, originating from lack of knowledge of parameters or by unpredictable effects of the environment. We focus on population processes, encompassing a large class of systems, from queueing networks to epidemic spreading. We set up a formal framework for imprecise stochastic processes, where some parameters are allowed to vary in time within a given domain, but with no further constraint. We then consider the limit behaviour of these systems as the population size goes to infinity. We prove that this limit is given by a differential inclusion that can be constructed from the (imprecise) drift. We provide results both for the transient and the steady state behaviour. Finally, we discuss different approaches to compute bounds of the soobtained differential inclusions, proposing an effective controltheoretic method based on Pontryagin principle for transient bounds. This provides an efficient approach for the analysis and design of large-scale uncertain and imprecise stochastic models. The theoretical results are accompanied by an in-depth analysis of an epidemic model and a queueing network. These examples demonstrate the applicability of the numerical methods and the tightness of the approximation.

show abstract

“…Thus, this traditional equilibrium selection criterion is unrelated to the dynamical equilibrium selection we presently consider. 4 See Boylan (1992) and Gilboa and Matsui (1992) for justification of random matching in a countably infinite population and Alos-Ferrer (1999) for the case of a continuum of agents.…”

mentioning

confidence: 99%

“…With the appropriate rescaling of time, it would drop out of equation (5), producing the replicator dynamics of equation (3). With p = 0, the linear combination dynamics (4) reduces to the imitate the best dynamics (2).…”

mentioning

confidence: 99%

Why learning doesn’t add up: equilibrium selection with a composition of learning rules

Golman

2010

Int J Game Theory

View full text Add to dashboard Cite

In this paper, we investigate the aggregate behavior of populations of learning agents. We compare the outcomes in homogenous populations learning in accordance with imitate the best dynamics and with replicator dynamics to outcomes in populations that mix these two learning rules. New outcomes can emerge. In certain games, a linear combination of the two rules almost always attains an equilibrium that homogenous learners almost never locate. Moreover, even when almost all weight is placed on one learning rule, the outcome can differ from homogenous use of that rule.Thus, allowing even an arbitrarily small chance of using an alternative learning style can shift a population to select a different equilibrium.

show abstract

Stochastic Approximations and Differential Inclusions

Cited by 253 publications

References 21 publications

Asynchronous Stochastic Approximation with Differential Inclusions

Asynchronous Stochastic Approximation with Differential Inclusions

Mean Field Approximation of Uncertain Stochastic Models

Why learning doesn’t add up: equilibrium selection with a composition of learning rules

Contact Info

Product

Resources

About