Optimal structured feedback policies for ABR flow control using two-timescale SPSA

Abstract

Abstract-Optimal structured feedback control policies for rate-based flow control of available bit rate service in asynchronous transfer mode networks are obtained in the presence of information and propagation delays, using a numerically efficient two-timescale simultaneous perturbation stochastic approximation algorithm. Models comprising both a single bottleneck node and a network with multiple bottleneck nodes are considered. A convergence analysis of the algorithm is presented. Numerical experiments demonstrate fast convergence even in the presence of significant delays. We also illustrate performance comparisons with the well-known Explicit Rate Indication for Congestion Avoidance (ERICA) algorithm and describe another algorithm (based on ERICA) that does not require estimating available bandwidth (as in ERICA).Index Terms-Network of nodes, optimal structured feedback policies, rate-based ABR flow control, single bottleneck node, twotimescale SPSA.

Randomized Difference Two-Timescale Simultaneous Perturbation Stochastic Approximation Algorithms for Simulation Optimization of Hidden Markov Models

“…The proof of Theorem 4.1 (below) differs at several places from that of a similar result in [6] since as already stated earlier, the latter was shown for finite state spaces which is not the case here. Moreover, Theorem 4.1 here, also serves to show stability of the proposed scheme, instead of just being a step in the convergence analysis as in [6].…”

“…We develop these algorithms in this paper in the framework of hidden Markov models with an unbounded state space, and therefore stability issues are explicitly addressed. The convergence of SPSA-1 in the finite state setting of [7] was proven in [6]. The convergence analysis of SPSA-2 in any setting has not been shown earlier.…”

“…Finally, in [6], the algorithm SPSA-1 was analyzed in the context of rate based feedback flow control in available bit rate (ABR) service in asynchronous transfer mode (ATM) networks. However, because of the finite state setting there, questions about stability of the scheme did not arise.…”

“…Finally, Assumption (A3) is required to ensure that the system remains stable, and is a standard assumption. In [3], the algorithms of [4] and [5] were applied to a closed loop feedback control problem in asynchronous transfer mode (ATM) networks and all the above assumptions were verified for the system (also see [6] for verification of Assumptions (A1), (A2) and (A3) for a similar system).…”

Time-optimal network queue control: the case of a single congested node

“…However [1,2] also assume a linear operation for the queue at all times. [6] derives stochastic optimal control policies assuming Markovmodulated capacity variations. [20] designs robust controllers to handle uncertain and heterogeneous network delays, but assumes linear operation too.…”