Diffusion approximations for controlled weakly interacting large finite state systems with simultaneous jumps

Abstract

We consider a rate control problem for an N -particle weakly interacting finite state Markov process. The process models the state evolution of a large collection of particles and allows for multiple particles to change state simultaneously. Such models have been proposed for large communication systems (e.g. ad hoc wireless networks) but are also suitable for other settings such as chemical-reaction networks. An associated diffusion control problem is presented and we show that the value function of the N -particle controlled system converges to the value function of the limit diffusion control problem as N → ∞. The diffusion coefficient in the limit model is typically degenerate, however under suitable conditions there is an equivalent formulation in terms of a controlled diffusion with a uniformly non-degenerate diffusion coefficient. Using this equivalence, we show that near optimal continuous feedback controls exist for the diffusion control problem. We then construct near asymptotically optimal control policies for the N -particle system based on such continuous feedback controls. Results from some numerical experiments are presented.AMS 2000 subject classifications: Primary 60K35, 60H30, 93E20; secondary 60J28, 60J70, 60K25, 91B70.

Diffusion approximations for load balancing mechanisms in cloud storage systems

“…In particular, here the number of nodes approach ∞ and the tools for proving convergence come from martingale problems and Markov process theory. A similar scaling regime was considered in [7] for certain systems motivated by ad-hoc wireless network models introduced in [3]. A key simplifying feature there is that the state space of an individual queue is a finite set.…”

“…In Section 2 we give a precise mathematical formulation of our model and a statement of our main results. Specifically, Theorem 1 provides the convergence in probability of the empirical measure process in D([0, T] : S) to the unique solution of the ODE defined in (7). In Theorem 2 we present the main diffusion approximation result.…”

“…A key simplifying feature there is that the state space of an individual queue is a finite set. Consequently, the limit diffusion in [7] is finite dimensional and, thus, for diffusion approximations, classical convergence theorems from [20] and [24] can be invoked. In contrast, the queue length processes in this work are unbounded, taking values in N 0 , and thus one needs to study diffusion approximations in an infinite-dimensional state space, namely the Hilbert space 2 .…”

Rate control under heavy traffic with strategic servers

“…In the heavy traffic regime with a fixed number of servers, performance improvement using an O( √ e n ) control has been well studied (see for example [52,Chapter 9]). In a setting (that is quite different from the one considered here) where the number of nodes/servers approach ∞, numerical results that show performance improvement under O( √ e n ) controls can be found in [22]. In the model we consider, the rates can depend on the state of the individual queue, furthermore a particular queue's state is influenced by the remaining queue states through their empirical measure.…”

“…Mean field approximations for weakly interacting stochastic particles have a long history starting from the works of Boltzmann, McKean, Kac and others (see [66] and references therein). Even in the context of queuing systems and communication networks, there have been many works [37,7,48,69,20,40,14,18,41,5,22]. Another related branch is agent based models with mean-field interaction (but without strategic agents), see e.g.…”