Asymptotic Properties of Distributed and Communicating Stochastic Approximation Algorithms

Kushner, Harold J.; Yin, Gang

doi:10.1137/0325070

Cited by 110 publications

(101 citation statements)

References 10 publications

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…Therefore we have a stochastic, asynchronous updating pattern, where a subset of an iterative process similar to (1.1) can be updated many times before the remaining components are selected for a single update. Based on this idea extensions to the standard theory have been examined such as those by Kushner and Yin [17,18]. Here however we follow the extension to asynchronous stochastic approximation provided by Borkar [9] and Konda and Borkar [14].…”

mentioning

confidence: 99%

“…Studying the performance of these processes can be carried out using the asynchronous stochastic approximation framework. However, the previous work in this area has focused on continuous, single-valued updates as discussed in the literature (see for example [9,14,17,18,24]). Furthermore some of the assumptions which are typically used are challenging to verify.…”

mentioning

confidence: 99%

“…Commonly known as the ODE method of stochastic approximation, originally proposed by Ljung [20], this technique has been extended by numerous authors, for example Benaïm [3], Benaïm, Hofbauer and Sorin [5], Borkar [11], Kushner and Clark [16] and Kushner and Yin [17,18]. In particular Benaïm, Hofbauer and Sorin [5] have developed the approach so that under some weak criteria {x n } n∈N can be updated via a set-valued mean field, F (·).…”

mentioning

confidence: 99%

See 2 more Smart Citations

Asynchronous Stochastic Approximation with Differential Inclusions

Perkins

Leslie

2012

Stochastic Systems

View full text Add to dashboard Cite

The asymptotic pseudo-trajectory approach to stochastic approximation of Benaïm, Hofbauer and Sorin is extended for asynchronous stochastic approximations with a set-valued mean field. The asynchronicity of the process is incorporated into the mean field to produce convergence results which remain similar to those of an equivalent synchronous process. In addition, this allows many of the restrictive assumptions previously associated with asynchronous stochastic approximation to be removed. The framework is extended for a coupled asynchronous stochastic approximation process with set-valued mean fields. Two-timescales arguments are used here in a similar manner to the original work in this area by Borkar. The applicability of this approach is demonstrated through learning in a Markov decision process.

show abstract

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Asynchronous Stochastic Approximation with Differential Inclusions

Perkins

Leslie

2012

Stochastic Systems

View full text Add to dashboard Cite

show abstract

“…Convergence of this distributed learning scheme is currently under investigation. More complicated distributed learning scenarios that incorporate partial information sharing between users (see, e.g., Kushner and Yin 1987) might be similarly considered.…”

Section: Discussionmentioning

confidence: 99%

Individual Equilibrium and Learning in Processor Sharing Systems

Inria

Shimkin

1998

Operations Research

View full text Add to dashboard Cite

We consider a processor-sharing service system, where the service rate to individual customers decreases as the load increases. Each arriving customer may observe the current load and should then choose whether to join the shared system. The alternative is a constant-cost option, modeled here for concreteness as a private server (e.g., a personal computer that serves as an alternative to a central mainframe computer). The customers wish to minimize their individual service times (or an increasing function thereof). However, the optimal choice for each customer depends on the decisions of subsequent ones, through their effect on the future load in the shared server. This decision problem is analyzed as a noncooperative dynamic game among the customers. We first show that any Nash equilibrium point consists of threshold decision rules and establish the existence and uniqueness of a symmetric equilibrium point. Computation of the equilibrium threshold is demonstrated for the case of Poisson arrivals, and some of its properties are delineated. We next consider a reasonable dynamic learning scheme, which converges to the symmetric Nash equilibrium point. In this model customers simply choose the better option based on available performance history. Convergence of this scheme is illustrated here via a simulation example and is established analytically in subsequent work.

show abstract

“…Under quite general conditions, constrained Q-learning algorithms were considered in Chap. 12 of [5] and the convergence was proved via stochastic approximation methods for distributed and asynchronous recursive procedures ( [6] and [7]). In particular, we demonstrated that how one may deal with constraints such as a hypercube [−B, B] r , where r is the dimension of the state variable.…”

Section: Q-learning Algorithmsmentioning

confidence: 99%

Q-Learning Algorithms with Random Truncation Bounds and Applications to Effective Parallel Computing

Yin

Wang

2007

J Optim Theory Appl

Self Cite

View full text Add to dashboard Cite

Motivated by an important problem of load balancing in parallel computing, this paper examines a modified algorithm to enhance Q-learning methods, especially in asynchronous recursive procedures for self-adaptive load distribution at runtime. Unlike the existing projection method that utilizes a fixed region, our algorithm employs a sequence of growing truncation bounds to ensure the boundedness of the iterates. Convergence and rates of convergence of the proposed algorithm are established. This class of algorithms has broad applications in signal processing, learning, financial engineering, and other related fields.

show abstract

Asymptotic Properties of Distributed and Communicating Stochastic Approximation Algorithms

Cited by 110 publications

References 10 publications

Asynchronous Stochastic Approximation with Differential Inclusions

Asynchronous Stochastic Approximation with Differential Inclusions

Individual Equilibrium and Learning in Processor Sharing Systems

Q-Learning Algorithms with Random Truncation Bounds and Applications to Effective Parallel Computing

Contact Info

Product

Resources

About