Control Techniques for Complex Networks

Meyn, Sean

doi:10.1017/cbo9780511804410

Cited by 264 publications

(232 citation statements)

References 159 publications

Supporting

Mentioning

228

Contrasting

Unclassified

Order By: Relevance

“…In these models the network load can be expressed as the maximum of ρ s over those s satisfying o s = 1 (see [34,Theorem 6.1.1]). In communication models without routing, the region R is precisely the multipleaccess rate region.…”

Section: Fluid Modelsmentioning

confidence: 99%

“…To obtain an approximation to this optimal control problem we relax the constraints on v(t) in the fluid model. Details of this approach can be found in [29,[32][33][34], and will be reviewed briefly in the following examples.…”

Section: Fluid Modelsmentioning

confidence: 99%

“…A relaxation for the stochastic model is defined similarly [34]. There are many results establishing solidarity between a relaxation and the original stochastic model.…”

Section: Multiple-access Modelmentioning

confidence: 99%

“…The starting point is a relaxation technique introduced in the thesis of Neil Laws [27] to obtain lower bounds on achievable delay performance for stochastic networks. 1 An analogous relaxation can be used to construct a simple model for control design [34]. This however requires a sequence of steps:…”

Section: Introductionmentioning

confidence: 99%

“…We borrow ideas from the MaxWeight policies of Tassiulas and Ephremides [16,37,39,40] and state space collapse from heavy traffic theory [4,23] to obtain a simple and effective approach to step 3. This approach to policy translation was introduced in [29], and developed further in [34]. The main ideas are summarized in Sect.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Coding and control for communication networks

et al. 2009

Self Cite

View full text Add to dashboard Cite

The purpose of this paper is to survey techniques for constructing effective policies for controlling complex networks, and to extend these techniques to capture special features of wireless communication networks under different networking scenarios. Among the key questions addressed are:(i) The relationship between static network equilibria, and dynamic network control. (ii) The effect of coding on control and delay through rate regions. (iii) Routing, scheduling, and admission control.Through several examples, ranging from multiple-access systems to network coded multicast, we demonstrate that the rate region for a coded communication network may be approximated by a simple polyhedral subset of a Euclidean space. The polyhedral structure of the rate region, determined by the coding, enables a powerful workload relaxation method that is used for addressing complexity-the relaxation technique provides approximations of a highly complex network by a far simpler one.These approximations are the basis of a specific formulation of an h-MaxWeight policy for network routing. Simulations show a 50% improvement in average delay performance as compared to methods used in current practice.

show abstract

Section: Fluid Modelsmentioning

confidence: 99%

Section: Fluid Modelsmentioning

confidence: 99%

“…A relaxation for the stochastic model is defined similarly [34]. There are many results establishing solidarity between a relaxation and the original stochastic model.…”

Section: Multiple-access Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Coding and control for communication networks

et al. 2009

Self Cite

View full text Add to dashboard Cite

show abstract

Lambda‐Policy Iteration: A Review and a New Implementation

Bertsekas

2012

Reinforcement Learning and Approximate Dynamic Programming for Feedback Control

View full text Add to dashboard Cite

In this paper we discuss λ-policy iteration, a method for exact and approximate dynamic programming. It is intermediate between the classical value iteration (VI) and policy iteration (PI) methods, and it is closely related to optimistic (also known as modified) PI, whereby each policy evaluation is done approximately, using a finite number of VI. We review the theory of the method and associated questions of bias and exploration arising in simulation-based cost function approximation. We then discuss various implementations, which offer advantages over well-established PI methods that use LSPE(λ), LSTD(λ), or TD(λ) for policy evaluation with cost function approximation. One of these implementations is based on a new simulation scheme, called geometric sampling, which uses multiple short trajectories rather than a single infinitely long trajectory. INTRODUCTIONApproximate dynamic programming (DP for short) has attracted substantial research interest, and has a wide range of applications, because of its potential to address large and complex problems that may not be treatable in other ways. The literature on the subject is very extensive, and includes several textbooks, research monographs, and surveys that relate to the computational context of this paper. For a nonexhaustive list, we mention the books by Bertsekas and Tsitsiklis [BeT96], Sutton and Barto [SuB98], Gosavi [Gos03], Cao [Cao07], Chang, Fu, Hu, and Marcus [CFH07], Meyn [Mey07], Powell [Pow07], Borkar [Bor08], Haykin [Hay08], Busoniu, Babuska, De Schutter, and Ernst [BBD10], and the author's text in preparation [Ber11a]; the edited volumes and special issues by White and Sofge [WhS92], Si, Barto, Powell, and Wunsch [SBP04], Lewis, Lendaris, and Liu [LLL08], and the 2007-2009 Proceedings of the IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning; and the recent surveys by Borkar [Bor09], Lewis and † are due to Bruno Scherrer for helpful comments, to Huizhen Yu for related collaboration, and to Mengdi Wang for assistance with computational experimentation. Vrabie [LeV09], Werbos [Wer09], Szepesvari [Sze10], and Bertsekas [Ber11b].The purpose of this paper is to critically review and extend a class of methods for exact and approximate DP, which are based on the λ-policy iteration (λ-PI) method, proposed by Bertsekas and Ioffe [BeI96]. This method is intermediate between the classical value iteration (VI) and policy iteration (PI) methods, and it is closely related to optimistic (also known as modified) PI, whereby each policy evaluation is done approximately, using a finite number of VI. It was originally used as the starting point for the development of approximate simulation-based DP methods of the temporal difference (TD) type, such as LSPE(λ) (see [BeI96], and also [BeT96], Sections 2.3.1 and 8.3). The emphasis in this paper is on implementations of λ-PI, which provide alternatives to approximate PI methods that use other more established methods for policy evaluation.We will focus on the α-discounted n-state Markovian Decisi...

show abstract

Feature Selection for Neuro‐Dynamic Programming

Huang

Chen

Mehta

et al. 2012

Reinforcement Learning and Approximate Dynamic Programming for Feedback Control

View full text Add to dashboard Cite

Neuro-Dynamic Programming encompasses techniques from both reinforcement learning and approximate dynamic programming. Feature selection refers to the choice of basis that defines the function class that is required in the application of these techniques.This chapter reviews two popular approaches to neuro-dynamic programming, TDlearning and Q-learning. The main goal of the chapter is to demonstrate how insight from idealized models can be used as a guide for feature selection for these algorithms. Several approaches are surveyed, including fluid and diffusion models, and the application of idealized models arising from mean-field game approximations. The theory is illustrated with several examples.

show abstract

Control Techniques for Complex Networks

Cited by 264 publications

References 159 publications

Coding and control for communication networks

Coding and control for communication networks

Lambda‐Policy Iteration: A Review and a New Implementation

Feature Selection for Neuro‐Dynamic Programming

Contact Info

Product

Resources

About